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PRELIMINARY AMENDMENT 

Sir: 

Please amend the above-identified patent application as follows: 
In the Specification; 

Please replace the Specification of the present application, including the Abstract, with 
the following Substitute Specification: 

SPECIFICATION 
TITLE OF THE INVENTION 
THREE-DIMENSIONAL MODEL PROCESSING APPARATUS, THREE-DIMENSIONAL 
MODEL PROCESSING METHOD AND PROGRAM PROVIDING MEDIUM 

BACKGROUND OF THE INVENTION 
This present invention generally relates to three-dimensional model processing. More 
specifically, the present invention relates to a three-dimensional model processing apparatus, a 
three-dimensional model processing method and a program providing medium implementing 
three-dimensional model processing. 

In the field of computer graphics, various techniques have been developed and realized 
with respect to apparatuses or methods for displaying a three-dimensional graphic on a display to 
execute processing to modify the orientation, shape, or the like of the displayed three- 
dimensional graphic. One such technique includes a method of inputting shape data of an object 
including depth data by using, for example, a mouse or a two-dimensional tablet. Further, there 
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is also used a method of acquiring data of actual three-dimensional objects by a three- 
dimensional digitizer or a three-dimensional scanner to acquire display data. In addition, as a 
method of carrying out processing such as deformation of an object displayed on a display, there 
is known a configuration using an input unit composed of a glove type manipulator. 

An example of the configuration of a shape change processing apparatus using the glove 
type manipulator is shown in Figure 1. In Figure 1, an operator dons a head mounted display 10 
to observe a display object 30. Further, a glove type manipulator 1 1 is fitted over or attached to 
the operator's hand. 

Pressure sensors or magnetic sensors for detecting movements of the operator's hand and 
fingers are attached to the manipulator 11. A detection signal is input to a control unit 20 
through an I/O interface 22. A CPU 21 executes processing in accordance with a program stored 
in a ROM 24 or a RAM 25, and executes processing to modify display parameters relating to the 
display object 30 in accordance with the detection signal. Display object 30 is updated based on 
the new display parameters through display control means 23. 

For example, when an operator executes an operation to "shave" the side surface of the 
display object 30 by using the glove type manipulator 1 1, a sensor attached to the manipulator 1 1 
detects the shaving operation of the manipulator 11, and the CPU 21 changes the display 
parameters relating to the display object 30 on the basis of the detection signal input through the 
I/O interface 22. Once the display object 30 is updated, the head mounted display 10 displays 
the display object 30 including the shaved portion 31. 

As stated above, in conventional three-dimensional modeling systems, various input 
techniques are used. In a previously described processing system using a mouse or a two- 
dimensional tablet, processing is input using two-dimensional information. Therefore, a feeling 
of restriction and/or disagreement may exist with respect to the operation of the three- 
dimensional object. Further, various processing such as movement, deformation, cutting, and the 
like with respect to the displayed object must be carried out by using only a single editing tool, 
for example, a mouse. This may result in the operator having difficulty intuitively grasping the 
proper setting of the editing tool. 

Input data from a three-dimensional digitizer and/or a three-dimensional scanner are 
useful for entering the shape of virtual objects, but are not suitable for deformation processing 
and the like of three-dimensional objects. Furthermore, such input units are very expensive. 
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It is desirable that input processing resulting from manipulation of a glove type 
manipulator is intuitive to the operator. However, in practice, initialization processing is 
required before actual manipulation processing may be carried out. For example, a "push" 
operation or a "pull" operation, with respect to the object, may be required to initialize the 
settings of the glove type manipulator. This may result in a drawback that operating the glove 
type manipulator may be difficult for a user who is unfamiliar with the glove type units. 
Moreover, these glove type input units are also very expensive. 

SUMMARY OF THE INVENTION 

An advantage of the present invention is that it provides a three-dimensional model 
processing apparatus, a three-dimensional model processing method and a program providing 
medium for storing instructions for implementing a three-dimensional model processing method 
in which an operator operates respective tools in a form closer to more practical processing by 
using virtual objects and various tools corresponding to respective processing, thereby making it 
possible to execute various processing of objects such as shape change, surface coloring, and the 
like. 

To this end, in an embodiment according to the present invention, a three-dimensional 
model processing apparatus is provided. The three-dimensional model processing apparatus 
includes a display unit for three-dimensionally displaying an object and an object tool 
representing the displayed object, wherein the object tool is capable of three-dimensional 
movement such that the position and the orientation of the object tool can be changed. Also 
included are an editing tool in which relative distance with respect to the object tool can be 
changed and a processor for detecting the relative position between the object tool and the 
editing tool. Preferably, the processor is configured to execute processing determined by the 
editing tool to modify attribute information of the displayed object on the basis of the detected 
relative positional information. 

In an embodiment, the processor is further configured to detect position information of 
the object tool based on changes in the position and the orientation of the object tool. Preferably, 
this detected position information is used to modify the attribute information of the displayed 
object. In an embodiment, the processor is further configured to execute processing 
corresponding a number of different types of editing tools. In an embodiment, the detected 
relative position information includes at least one of the relative distance between the object tool 
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and the editing tool and the relative angle between the object tool and the editing tool. 
Preferably, the attribute information of the displayed object is attribute information relating to at 
least one of shape, color and sound. Further, the processor is configured to execute functional 
operation of the displayed object as processing determined by the editing tool 

In another embodiment according to the present invention, a three-dimensional model 
processing method is provided. Also provided are an object that is three-dimensionally 
displayed on a display unit, an object tool representing the displayed object and an editing tool. 
The object tool is capable of three-dimensional movement such that the position and the 
orientation of the object tool can be changed and the editing tool is capable of three-dimensional 
movement such that the relative position between the editing tool and the object tool can be 
changed. The three-dimensional model processing method includes the steps of detecting the 
relative position between the object tool and the editing tool and executing processing 
determined by the editing tool to modify attribute information of the displayed object on the 
I basis of the detected relative position information. 

In an embodiment, the three-dimensional model processing method further includes the 
steps of detecting position information of the changed object tool based on changes in the 
I position and the orientation of the object tool and modifying the attribute information of the 
displayed object based on the detected position information. In an embodiment, the three- 
dimensional model processing method further includes the steps of selecting the editing tool 
I from a plurality of different types of editing tools and executing processing corresponding to the 
type of editing tool selected. 

In an embodiment, detected relative position information is at least one of the relative 
distance between the object tool and the editing tool and the relative angle between object tool 
and editing tool. In an embodiment, the attribute information of the displayed object is attribute 
information relating to at least one of shape, color and sound. In a further embodiment, the 
three-dimensional model processing method includes the step of executing functional operation 
of the displayed object as processing determined by the editing tool. 

In yet another embodiment according to the present invention, a computer readable 
medium storing a computer readable program is provided. Also provided are an object that is 
three-dimensionally displayed on a display unit, an object tool representing the displayed object 
and an editing tool. The object tool is capable of three-dimensional movement such that the 
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position and the orientation of the object tool can be changed and the editing tool is capable of 
three-dimensional movement such that the relative position between the editing tool and the 
object tool can be changed. The computer readable program includes the steps of detecting the 
relative position between the object tool and the editing tool and executing processing 
determined by the editing tool to modify attribute information of the displayed object on the 
basis of the detected relative position information. 

Additional features and advantages of the present invention are described in, and will be 
apparent from, the following Detailed Description of the Invention and the Figures. 

BRIEF DESCRIPTION OF THE FIGURES 

Figure 1 is a schematic diagram showing a conventional three-dimensional model 
processing configuration using a glove type manipulator. 

Figure 2 is a schematic diagram showing an embodiment of a three-dimensional model 
processing apparatus according to the present invention. 

Figure 3 is a schematic diagram illustrating processing in a three-dimensional model 
processing apparatus according to an embodiment of the present invention. 

Figure 4 is a block diagram showing an embodiment in which a tablet is used as an input 
unit in a three-dimensional model processing apparatus according to the present invention. 

Figure 5 is a block diagram showing an embodiment in which magnetic sensors are used 
in an input unit in a three-dimensional model processing apparatus according to the present 
invention. 

Figure 6 is a block diagram showing an embodiment in which ultrasonic sensors are used 
in an input unit in a three-dimensional model processing apparatus according to the present 
invention. 

Figure 7 is a block diagram showing an embodiment in which ultrasonic transponders are 
used in an input unit in a three-dimensional model processing apparatus according to the present 
invention. 

Figure 8 is a flowchart illustrating processing in a three-dimensional model processing 
apparatus according to an embodiment of the present. 

Figure 9 is a flowchart illustrating processing where a trowel is used as an editing tool in 
a three-dimensional model processing apparatus according to an embodiment of the present 
invention. 
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Figure 10 is a data flow diagram illustrating processing where a trowel is used as an 
editing tool in a three-dimensional model processing apparatus according to an embodiment of 
the present invention. 

Figure 1 1 is a flowchart illustrating processing where a pinch is used as an editing tool in 
a three-dimensional model processing apparatus according to an embodiment of the present 
invention. 

Figure 12 is a data flow diagram illustrating processing where a pinch is used as an 
editing tool in a three-dimensional model processing apparatus according to an embodiment of 
the present invention. 

Figure 13 is a flowchart illustrating processing where a brush is used as an editing tool in 
a three-dimensional model processing apparatus according to an embodiment of the present 
invention. 

Figure 14 is a data flow diagram illustrating processing where a brush is used as an 
editing tool in a three-dimensional model processing apparatus according to an embodiment of 
the present invention. 

Figure 15 is a flowchart illustrating processing where a spray is used as an editing tool in 
a three-dimensional model processing apparatus according to an embodiment of the present 
invention. 

Figure 16 is a data flow diagram illustrating processing where a spray is used as an 
editing tool in a three-dimensional model processing apparatus according to an embodiment of 
the present invention. 

Figure 17A is a perspective view showing an action area in an embodiment where a spray 
is used as an editing tool in a three-dimensional model processing apparatus according to the 
present invention, and Figure 17B is a perspective view showing a coloring range in an 
embodiment where spray is used as an editing tool. 

Figures 18 A, 18B and 18C are perspective views showing embodiments of a three- 
dimensional model processing apparatus according to the present invention. 

Figure 19 is a perspective view showing an embodiment of a three-dimensional model 
processing apparatus according to the present invention. 
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DETAILED DESCRIPTION OF THE INVENTION 
Figure 2 is a schematic diagram showing an embodiment of a three-dimensional model 
processing apparatus according to the present invention. The three-dimensional model 
processing apparatus 100 includes a central processing unit (CPU) 101 for executing a 
processing program, a program memory 102 into which a processing program is stored, and a 
data memory 103 in which processing data and information such as attribute data of position, 
orientation, shape, color, and the like of the object to be edited or attribute data of the editing 
tool, and the like are stored. Also included are a frame memory 104 in which picture 
information for displaying, on a picture display unit 110, the object to be edited, the editing tool, 
an instruction to user, and the like are stored, an input unit 105 for inputting various instructions 
with respect to the object to be edited, an external storage unit 106 for storing observation 
information or a processing result, and a bus 107 for permitting data transfer between the 
respective units. The picture display unit 110 is connected through the frame memory 104. 
Thus, picture data stored in the frame memory 104 may be displayed. 

The input unit 105 includes a number of tools for carrying out various processing such as 
I deformation, surface color change, and the like with respect to the object to be edited (i.e., the 
object being displayed on the display unit or the displayed object). 

A three-dimensional model processing apparatus according to an embodiment of the 
I present invention includes, as shown in Figure 3, an object tool 301 corresponding to the 
displayed object and editing tools 302 to 306 for implementing various processing to the 
displayed object. The object tool 301 and the editing tools 302 to 306 are also displayed on a 
display 307. By using these tools, processing such as deformation, coloring, and the like with 
respect to displayed object 308 may be executed. 

As shown in Figure 3, the editing tools 302 to 306 include a push tool 302 having, for 
example, having the shape of a trowel for executing a "push" operation with respect to the 
displayed object 308; a pull tool 303 having the shape of a pinch or tongs for executing a "pull" 
operation with respect to the displayed object 308; a brush tool 304 having, for example, the 
shape of a brush for executing a "fine coloring" operation with respect to the displayed object 
308; a spray tool 305 having, for example, the shape of a can of spray paint for executing a 
"rough coloring" operation with respect to the displayed object 308; and a cut tool 306 having, 
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for example, the shape of a knife for executing a "cut off, separate" operation with respect to the 
displayed object 308. 

The operator operates the editing tools 302 to 306 in a predetermined action area (e.g., a 
sensor detection area) as indicated by action areas 201 to 204 in Figure 3, thereby causing a 
change in distance between the object tool 301 and the editing tools 302 to 306 and a change in 
the angle therebetween. As a result, processing corresponding to the respective tools, that is, 
deformation of the displayed object 308 corresponding to the "push" operation by the push tool 
302 and the "pull" operation by the pull tool 303, and/or change of the surface color of the 
displayed object 308 corresponding to the brush tool 304 and the spray tool 305 are implemented 
to the displayed object 308. 

In an embodiment, a clay model is used as the object tool 301 and is subjected to 
operations such as deformation, coloring, or the like. Information is input from a sensor for 
detecting the position and the orientation of the clay model in the three-dimensional space. The 
object tool 301 may be a three-dimensional shape as shown in Figure 3, for example, a spherical 
shape or a shape approximate to the displayed object 308. The object tool 301 is configured such 
that the operator can hold it by one hand such that it may be freely moved or rotated. 

The editing tool includes, as a parameter, information input from a sensor for detecting 
the position and the orientation of the editing tool in the three-dimensional space. The editing 
tool has a shape that operators can intuitively associate with a processing mode in accordance 
with the respective processing. For example, as shown in Figure 3, the editing tool can be in the 
shape of a trowel as a push tool 302, the shape of a pinch or tongs as a pull tool 303, the shape of 
a brush as a brush tool 304, the shape of a can of spray paint as a spray tool 305, and the shape of 
a knife as a cut tool 306. 

The object tool 301 and the editing tools 302 to 306 correspond to the input unit 105 in 
Figure 2. An embodiment of the input unit will be described with reference to Figures 4 to 7. It 
is to be noted that, in Figures 4 to 7, the editing tools 302 to 306 are divided into a tool for 
deformation processing and a tool for color processing without classifying the type of tool being 
used. 

Figure 4 is an embodiment where an input unit includes a tablet 404. Position 
information and orientation information of an object tool 401, a deformation tool 402 and a 
coloring tool 403 are detected by input through the input tablet 404. A detection function by a 
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chip, a coil, or the like is added to the object tool 401, the deformation tool 402 and the coloring 
tool 403 similarly to an input pen associated with a conventional tablet. Thus, the position and 
the orientation information of the respective tools may be detected. For example, coils having 
different detection values are provided on the respective surfaces 1 to 6 of the object tool 301 of 
hexahedron shape as shown in Figure 3. Thus, when the different detection values of the coils 
are obtained via the tablet 404, mounting surfaces are identified, thereby making it possible to 
obtain three-dimensional information from these mounting surfaces. Alternatively, a 
configuration may be employed in which Z-axis information obtained from other sources, such 
as an optical sensor, a magnetic sensor or the like is combined with XY axis (i.e., two- 
dimensional) information obtained from the tablet 404 to identify a three-dimensional position 
and to obtain three-dimensional data. 

Figure 5 is an embodiment where an input unit includes magnetic sensors. The area 
p within a magnetic field produced by a magnetic source 504 is an action area, and operations by 
H an object tool 501, a deformation tool 502 and a coloring tool 503 are carried out within the 
'ft action area. Magnetic sensors are respectively attached to the object tool 501, the deformation 
\1 tool 502 and the coloring tool 503. These magnetic sensors output magnetic displacement data 
iaj[ 1 to dedicated interface hardware 505 within the action area. The dedicated interface hardware 

Q 505 calculates position information data and orientation information data of the respective tools 

! f | 

pi I on the basis of this data. In an embodiment, the dedicated interface hardware 505 shown m 
Figure 5 may be replaced by a calculation processing software program. 

J «s=| 

ffj Figure 6 is an embodiment where an input unit includes ultrasonic sensors. The area 

where ultrasonic waves produced by an ultrasonic transmitting section 604 can be detected is an 
action area, and operations by an object tool 601, a deformation tool 602 and a coloring tool 603 
are carried out within the action area. Ultrasonic sensors are respectively attached to the object 
tool 601, the deformation tool 602 and the coloring tool 603. These ultrasonic sensors detect 
arrival times from the source of an ultrasonic wave or the interference of ultrasonic waves and 
the like within the action area. Dedicated interface hardware 605 calculates position information 
data and orientation information source data of the respective tools on the basis of the detected 
data. In an embodiment, the dedicated interface hardware 605 shown in Figure 6 may be 
replaced by a calculation processing software program. 
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Figure 7 is an embodiment where an input unit includes an ultrasonic sensor. An 
ultrasonic transmitting/receiving section 704 transmits and receives ultrasonic waves. The area 
where ultrasonic waves produced by the ultrasonic transmitting/receiving section 704 can be 
detected is an action area. Processing operations by an object tool 701, a deformation tool 702 
and a coloring tool 703 are carried out within the action area. Transponders having a function to 
receive ultrasonic waves produced by the ultrasonic transmitting/receiving section 704 and to 
send back ultrasonic waves are respectively attached at the object tool 701, the deformation tool 
702 and the coloring tool 703. The ultrasonic transmitting/receiving section 704 receives 
ultrasonic waves sent back from the respective tools to detect a transmission time of the 
ultrasonic waves, interference of the ultrasonic waves, or the like to calculate position 
information data and orientation information source data of the respective tools on the basis of 
this data. 

The input unit 105 shown in Figure 2 is realized as an input device using various sensors 
as described above. In an embodiment, information input/output with respect to a three- 
dimensional model processing apparatus 100 from sensors of the respective tools may be 
configured as a data line of wire or a wireless data transmitting/receiving configuration. In an 
1 embodiment, the input unit 105 may include a combination of the various sensors. 

In an embodiment, a random accessible memory medium such as a hard disk drive 
(HDD), an optical disk, or the like is desirable as the external storage unit 106 shown in Figure 2. 
Additionally, the external memory unit may include a tape streamer or non-volatile 
semiconductor memory represented by a memory stick. Also, it is possible to utilize an external 
storage unit of another network-connected system. In an embodiment, the central processing 
unit (CPU) 101 executes processing in accordance with a processing program recorded in the 
program memory 102. 

The flow of processing using the three-dimensional model processing apparatus shown in 
Figure 2 will be described with reference to the flowchart shown in Figure 8. The flowchart 
shown in Figure 8 illustrates processing executed in accordance with a processing program 
recorded in the program memory 102. In Figure 8, an embodiment will be explained using at 
least one of a push tool 302, a pull tool 303, a brush tool 304 (i.e., a coloring tool) and a spray 
tool 305 as an editing tool. In an embodiment, it is possible to use various types of editing tools 
not listed above for executing various types of processing. 



10 



Appl. No. PCT/JP01/03328 

Initially, at step S801, data of a position and an orientation in the three-dimensional space 
of the object to be edited (i.e., the displayed object) are acquired. This data corresponds to 
attribute data such as position, orientation, shape, color, and the like of the displayed object and 
is stored in the data memory 103 (shown in Figure 2). An initial position of the object tool 301 
(shown in Figure 3) is set as a position and an orientation corresponding to this attribute data. 
Thereafter, the operator moves the object tool 301 within the action area (i.e., the area where 
position detection by a sensor can be made), the position and the orientation of the object tool are 
detected by a sensor output of the object tool 301, and the attribute data of the displayed object is 
rewritten in accordance with the movement. Thus, the appearance displayed object is updated on 
the picture display unit 1 10 on the basis of the rewritten attribute data. 

Then, at step S802, a check is performed to determine whether the object tool and the 
editing tool are within the action area. The action area is the detectable area of the various 
sensors used as the input unit. As discussed above, various input units can be used in accordance 
with various embodiments of the present invention. For example, a tablet can be used as the 
input unit and the action area is the area of the tablet, a magnetic sensor can be used as the input 
unit and the action area is the detectable area of the magnetic sensor, or an ultrasonic sensor can 
be used as the input unit and the action area is the detectable area of the ultrasonic sensor. 

When it is determined at step S803 that the tool is not within the action area, processing 
jumps to step S814. When it is determined that tool is within the action area, processing 
proceeds to step S804 to determine the type or kind (ID) of tool being used. The determination 
of the tool type is determined with respect to the type of editing being performed, such as 
deformation, coloring, and the like. With respect to the types of editing tools, the operator may 
designate the tool to input identification data (ID) indicating the tool type, or may employ a 
configuration for outputting an identification signal (ID) from the editing tool within the action 
area to identify the type of tool on the basis of the identification signal. 

When the type of editing tool is determined at the step S804, processing proceeds to steps 
S805 to S812. Step S805 to the step S806 illustrates processing where the editing tool is a push 
tool, step S807 to the step S808 illustrate processing where the editing tool is a pull tool, step 
S809 to the step S810 illustrate processing where the editing tool is a brush tool, and step S811 
to the step S812 illustrate processing where the editing tool is a spray tool. The processing 
illustrated by steps S806, S808, S810, S812 corresponds to subroutines of the respective tools, 
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and are called when the editing tool is specified. Where the editing tool does not correspond to a 
known type of editing tool, processing proceeds to step S813, whereby a warning is produced, 
for example, displaying a warning message on the picture display unit 110 or outputting a 
warning sound. Explanation will be given below with reference to Figures 9 to 16 in connection 
with the respective processing of steps S806, S808, S810 and S812 (i.e., subroutine processing 
corresponding to the respective editing tools). 

At step S814, the respective editing tools are displayed on the picture display unit 110, 
and display of the object processed by the editing tool is carried out. The display modes (form) 
of the respective editing tools are modes based on shape data set in advance for each editing tool. 
For example, if the editing tool is a push tool, the shape of a trowel is displayed on the picture 
display unit 1 1 0 or if the editing tool is a spray tool, the shape of a can of spray paint is displayed 
on the picture display unit 110. Moreover, the displayed object is displayed as an object to 
which deformation, coloring or the like is applied by each of the editing tools. Namely, as 
attribute data such as position, orientation, shape, color, and the like of the displayed object is 
modified by the editing tools, these changes are reflected and displayed on the picture display 
| unit 1 10. Attribute data of the displayed object rewritten by the operator using the editing tools 
is stored into the data memory 103 (shown in Figure 2). 

Further, at step S815, a check is performed to determine if processing is completed. 
Processing may be completed by input from a user who is the operator, or from a rule prescribed 
within the processing application. For example, with a game program, processing may be 
completed by a completion command such as game over. In addition, the determination that 
processing is completed may be based on a hardware or software restriction such as memory full 
and the like. When it is determined at step S815 that processing is not completed, processing 
returns to the start of the processing flow to repeatedly execute similar processing. 

Figure 9 is a flowchart illustrating processing using a push tool to perform deformation of 
the displayed object. Step S901 is a step for obtaining position/orientation data on the three- 
dimensional space of a trowel being used as a push tool. As previously described, such data are 
data obtained on the basis of respective sensor outputs in accordance with the type of input unit 
utilized, for example, tablets, magnetic sensors, ultrasonic sensors or the like. 

In step S902, a space relationship/relative position between the displayed object and the 
editing tool is determined based on the position/orientation data of the displayed object (acquired 
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in step S801 of Figure 8) and the position/orientation data of the trowel as the editing tool 
(obtained in step S901). 

In step S903, a determination is made as to whether or not deformation of the displayed 
object by the trowel as the editing has been carried out. Put more simply, step 903 serves to 
determine whether or not position data of the editing tool as detected by various sensors overlaps 
with position data of the object tool (i.e., has the editing tool penetrated the displayed object). It 
is to be noted that it is not necessary that editing tools 302 to 306 actually penetrate the object 
tool 301. The apparatus may be configured to suppose that the editing has penetrated the 
displayed object if the distance between the object tool and the editing tool is less than a 
predetermined value. Alternatively, the object tool 301 may be composed of, for example, a 
deformable sponge shaped material such that the editing tool may actually penetrate the object 
tool 301. In an embodiment, deformation of the displayed object may be executed on the basis 
I of the actual position data of the editing tool and the object tool. 

Deformation processing of the object to be edited as displayed on the picture display unit 
110 at step S904 can be realized by moving positions of vertices of polygons when the display 
| data is polygons data. Where the distance between vertices is large, processing for generating 
new vertices is executed. Moreover, where the displayed object is represented as parametric 
curves, their control points are moved. Further, in the case where the displayed object is 
represented by voxels, deformation can be realized by (Boolean operation) processing to add 
data. The deformation processing is carried out in accordance with data mode (form). In either 
case, processing based on displacement data of the relative distance between object tool and 
editing tool is carried out. 

After deformation processing of the displayed object is executed in step S904, the 
rewritten attribute data of the displayed object including the deformed result is stored into the 
data memory 103 or external storage unit 106 (shown in Figure 2) in step S905. 

In order to supplement the explanation of the flowchart shown in Figure 9, explanation 
will be given with reference to the data flow diagram of Figure 10 using the trowel as a push 
tool. The data flow diagram is a view showing the flow of data extending from a data source 
within the object to a target within another object via a process for converting data. The data 
flow diagram includes processes (indicated as ellipses) for converting data, data flows (indicated 
as arrows) for carrying data, actor objects (indicated as rectangles) indicating production and 
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consumption of data, and data store objects (indication put between two lines) for passively 
carrying out storage of data. 

The data flow of Figure 10 illustrates processing using the trowel as a push tool 
Processing is started by operation of the object tool (sensor) 1001 as an actor object and the 
trowel tool (sensor) 1002 as an actor object. From the object tool 1001, orientation information 
and position information of the object are acquired. From the trowel tool 1002, position 
information of the tool is acquired. Further, from the shape information of the object which is 
attribute data of the object to be edited (i.e., the displayed object) which has been acquired in 
advance and the surface attribute information of the object, the orientation information and the 
position information of the object and the position information of the tool which are stored in the 
data memory 103 (shown in Figure 2), calculation processing 1003 to calculate the degree of 
deformation is executed. On the basis of this calculation, processing for changing the attribute 
information (i.e., deformation processing 1004) is executed. In combination with this 
processing, rendering processing 105 for generating a display picture with respect to the picture 
display unit 1 10 is executed. Thus, a rendering processing result is stored in the memory for the 
| display (i.e., the frame memory 104 of Figure 2) and is displayed on the picture display unit 1 10. 

Figure 1 1 is a flowchart illustrating processing using a pinch or tongs as a pull tool to 
perform deformation of the displayed object. Step SI 101 is a step for obtaining position data 
I and orientation data in the three-dimensional space of the pinch or tongs used as the pull tool. 
As previously described, such data are data obtained on the basis of respective sensor outputs m 
accordance with type of input unit utilized. 

In step SI 102, a determination is made as to whether or not a switch of the pinch is 
pushed down. The pinch used a pull tool in this embodiment is configured with a switch for 
determining the start point and the end point of the pull operation. The position of the pool tool 
at the point when the switch is pushed down is either the start point ("ON") or the end point 
("OFF") of the pull operation with respect to the displayed object. When it is determined in step 
SI 102 that the switch is not pushed down, processing for turning OFF "flag to show it was 
pushed down last" is executed at step SI 109 thereafter to execute processing to return to the end 
position (i.e., the flow start position). 

When it is determined at the step SI 102 that the switch has been pushed down, 
processing proceeds to step SI 103, at which point it is determined whether or not "flag to show it 
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was pushed down last" is turned ON. Where "flag to show it was pushed down last" is not 
turned "ON", it is judged that the push down of the switch at this time is "ON", (i.e., the current 
position is the start position of the pull operation). At step SI 108, processing for storing position 
information of the pull tool of switch-pushing-down-position into the data memory 103 to return 
to the flow start position is executed. 

When it is determined at the step SI 103 that "flag to show it was pushed down last" is 
turned ON, it is determined that the current switch pushing down of this time is "OFF" and this 
time period is the execution period of the pull operation. At step SI 104, position information of 
the switch pushing position (i.e., position information as the end position of the pull operation) is 
stored into the data memory 103. 

In the subsequent step SI 105, a movement quantity of the pinch used as the pull tool 
from the last switch-pushing-down-point to the switch-pushing-down-point of this time is 
calculated. This movement quantity is the effective processing period of the pull operation. In 
the subsequent step SI 106, deformation processing of the object is executed on the basis of the 
movement quantity determined at the step SI 105. At step SI 107, new attribute information 
obtained by deformation processing (i.e., shape data of the object is stored into the data 
memory). Display of the object is carried out on the picture display unit 110 on the basis of the 
new attribute information. 

In order to supplement the explanation of the flowchart shown in Figure 11, explanation 
will be given with reference to the data flow diagram of Figure 12 using the pinch as a pull tool. 
In Figure 12, processing is started by operation of the object tool (sensor) 1201 as an actor object 
and pinch tool (sensor) 1202 as an actor object. From the object tool 1201, orientation 
information and position information of the object are acquired. From the pinch tool 1202, 
position information of the tool and the last position information of the tool are acquired. This 
information corresponds to the start position and the end position of the pull operation by the 
pinch or the tongs. 

On the basis of the acquired position information of the tool, calculation processing 1203 
to calculate a movement quantity of the tool is executed. On the basis of the calculated 
movement quantity and position information and orientation information of the object, 
calculation processing 1204 to calculate a deformation quantity of the object is executed. 
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Further, on the basis of the calculation processing result of the deformation quantity, 
change processing to modify the attribute information (i.e., deformation processing 1205) is 
executed. In combination with this processing, rendering processing 1206 for generating a 
display picture with respect to the picture display unit 110 is executed. Thus, a rendering 
processing result is stored into the memory for the display (i.e., the frame memory 104 of Figure 
2) and is displayed on the picture display unit 1 10. 

Figure 13 is a flowchart illustrating processing using a brush tool as a coloring tool to 
apply coloring to the displayed object. Step S1301 is a step for obtaining position/orientation 
data on the three-dimensional space of the brush used as the coloring tool. As previously 
describe, such data are obtained on the basis of respective sensor outputs in accordance with the 

type of input unit utilized. 

In step SI 302, a space relationship/relative position between the displayed object and the 
editing tool is determined from the position/orientation data of the displayed object (acquired in 
step S801 of Figure 8) and the position/orientation data of the brush as the editing tool (obtained 
instepS1301). 

In step SI 303, a determination is made as to whether or not deformation of the displayed 
object by the brush as the editing tool brush has been carried out. The determination may be 
such that, for example, similarly to the previously described pull tool pinch, a switch is provided 
at the brush to detect the push-down of the switch. Alternatively, a configuration may be 
employed to carry out coloring by the brush when the distance between the displayed object and 
the editing tool (determined in step S1302) is less than a predetermined distance. 

At the subsequent step SI 304, coloring processing with respect to the object is executed. 
In an embodiment, the operator sets the color used in this step in advance. After coloring 
processing of the displayed object is executed, the rewritten attribute data of the displayed object 
including the colored result is stored in the data memory 103 or the external storage unit 106 
(shown in Figure 2) in step SI 305. 

In order to supplement the explanation of the flowchart shown in Figure 13, explanation 
will be given with reference to the data flow diagram of Figure 14 using the brush as a coloring 
tool. In the data flow of Figure 14, processing is started by operation of an object tool (sensor) 
1401 as actor object and a brush tool (sensor) 1402 as an actor object. From the object tool 
1401, orientation information and position information of the object are acquired. From the 
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brush tool 1402, position information of the tool is acquired. Further, from the shape 
information of the object which is the attribute data of the displayed object which has been 
acquired in advance and the orientation information and the position information of the object 
and the position information of the tool which are stored in the data memory 103 of Figure 2, 
calculation processing 1403 to calculate the coloring range is executed. On the basis of this 
calculation result and the surface attribute information of the object, processing for changing the 
attribute information (i.e., coloring processing 1404) is executed. In combination with this 
processing, rendering processing 1405 for generating a display picture with respect to the picture 
display unit 110 is executed, and a rendering processing result is stored into memory for the 
display (i.e., the frame memory 104 of Figure 2) and is displayed on the picture display unit 110. 

Figure 15 is a flowchart illustrating processing using a spray tool as a coloring tool to 
apply coloring to the displayed object. Step S1501 is step for obtaining position/orientation data 
in the three-dimensional space of the spray (i.e., the spray tool having the shape of a can of spray 
paint) used as a coloring tool. As previously described, such data are data obtained on the basis 
of respective sensor outputs in accordance with the type of input unit utilized. 

In step SI 502, the positional relationship of the action range between the displayed object 
and the spray tool is determined from position/orientation data of the displayed object (acquired 
in step S801 of Figure 8) and position/orientation data of the spray tool as the editing tool 
(obtained in step SI 501). 

In step SI 503, a determination is made as to whether or not the positional relationship of 
the action range between the displayed object and the spray tool (determined in step S1502) falls 
within the action area of the spray tool as the editing tool. In an embodiment, the area which can 
be colored by the spray tool is set in advance. For the displayed object to be colored, the 
positional relationship of action range between the displayed and the spray tool must fall within 
the action area of the spray tool as the editing tool. Thus, the action area of the spray tool is set 
in advance as attribute data of the editing tool. 

In an embodiment, the action area of the spray tool is set as a conical area 1701 in which 
predetermined points of the spray tool are caused to be vertices as indicated by Figure 17 A, for 
example, and a coloring range 1702 with respect to an object is set within the action area on the 
object 1704 surface as shown in Figure 17B. Push button 1703 is provided on the spray tool, and 
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the coloring range 1702 is set on the basis of the positional relationship at the point when the 
push button 1703 is pushed down. 

When it is determined at the step SI 503 that the corresponding area is outside the action 
area of the spray tool, coloring processing by the spray tool is not executed. As a result, 
processing is completed. Thus, the routine is returned to the start position. When it is determined 
in step SI 503 that the corresponding area is inside the action area of the spray tool, the action 
area is displayed on the picture display unit 1 10 at step SI 504. 

In step SI 505, the switch of the spray tool is pushed down, and a determination is made 
as to whether or not the switch is turned ON. The switch being ON initiates coloring processing 
with respect to the displayed object in step SI 506. In an embodiment, the operator sets the color 
used in this step in advance. After coloring processing of the displayed object is executed in step 
SI 506, rewritten attribute data of the displayed object including the colored result is stored into 
the data memory 103 or the external storage unit 106 (shown in Figure 2) in step SI 507. 

In order to supplement the explanation of the flowchart shown in Figure 15, explanation 
will be given with reference to the data flow diagram of Figure 16 using the spray tool as a 
coloring tool. In the data flow of Figure 16, processing is started by operation of the object tool 
(sensor) 1601 as actor object and the spray tool (sensor) 1602 as an actor object. From the object 
tool 1601, orientation information and position information of the object are acquired. From the 
spray tool 1602, position information and orientation information of the tool are acquired. On 
the basis of the position information and the orientation information of the spray tool, calculation 
processing 1603 to calculate the action area which can be colored by the spray tool is executed. 

Further, calculation processing 1604 to calculate the coloring range is carried out on the 
basis of the shape information of the object which is the attribute data of the displayed object 
which has been acquired in advance and the orientation information and the position information 
of the object and the calculated action area of the tool spray which are stored in the data memory 
103 of Figure 2. Processing to modify the attribute information (i.e., coloring processing 1605) 
is executed on the basis of this calculation. In combination with this processing, rendering 
processing 1606 for generating a display picture with respect to the picture display unit 110 is 
executed, and a rendering processing result is stored into the memory for the display (i.e., the 
frame memory 104 of Figure 2) and is displayed on the picture display unit 110. 
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As stated above, in accordance with a three-dimensional model processing apparatus 
according to an embodiment of the present invention, the operator uses the object tool 
corresponding to the displayed object and various dedicated tools to execute a change in the 
relative position between both tools, whereby the processing unit executes processing determined 
in advance by the dedicated tool selected. As a result, processing corresponding to the respective 
editing tools such as deformation, coloring, and the like (i.e., an attribute change) is carried out 
with respect to the displayed object, and the display of its result is executed. Accordingly, the 
operator can execute various processing with such a feeling to directly implement processing to 
the displayed object. 

Explanation will be given in connection with a practical embodiment of an application of 
a three-dimensional model processing apparatus according to the present invention. In an 
embodiment, a three-dimensional model processing apparatus according to the present invention 
as applied to on-line shopping using a communication network such as the internet or the like is 
shown in FIGS. 18Ato 18C. 

In an embodiment, a three-dimensional model processing apparatus of this invention is 
contained within, for example, personal computers of users to display, on display 1801, goods 
from a provider who provides on-line shopping. Such goods display may be carried out by 
storing three-dimensional picture data relating to the goods provided from the provider through a 
network into a storage unit such as a memory, a hard disk, and the like within the computer of 
the user to read out such data. Alternatively, goods display may be carried out by using data 
stored on a medium, for example, CD, DVD, or the like that the goods provider provides. 

The user who studies the goods displayed as a three-dimensional picture on the display 
1801 holds, by one hand, the object tool 1802 corresponding to the display goods and holds, by 
the other hand, the operation tool 1803 for executing various processing with respect to the 
goods. A user studying the goods can relatively operate the object tool 1802 and the operation 
tool 1803 to execute processing corresponding to respective operation tools with respect to the 
goods displayed on the display. For example, by changing the angle/orientation of the object 
tool 1802, the article displayed on display is displayed in the state where the angle/orientation is 
changed in a manner from Figure 18A to Figure 18B. Thus, the user can observe the goods or 
articles from all angles. 
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Further, for example, a previously described push tool may be used as the operation tool 
1803, and a push operation may be executed by the tool which pushes each operation button of 
the object tool corresponding to, for example, a video camera. Thus, as shown in Figure 18C, it 
is possible to employ such a configuration that an operation button of the video camera on the 
display 1801 is pushed down such that processing corresponding to the depressed operation 
button is executed on the display. 

In an embodiment, each operation button may be depressed and executed on the display 
by employing a configuration such that each switch position of the displayed object is specified 
on the basis of position information of the push tool to execute a processing program 
corresponding to the specified switch. By executing the program corresponding to the depressed 
switch, operation of the goods displayed in display 1801 may be carried out. In a manner stated 
above, a user can actually obtain a feeling of practically handling or holding an article and 
operating it by relative operation between the object tool and the operation tool. 

Further, explanation will be given in connection with a different practical embodiment of 
an application of a three-dimensional model processing apparatus according to the present 
invention. Figure 19 constitutes a three-dimensional model processing apparatus according to an 
embodiment of the present invention within, for example, a personal computer to display a 
virtual pet on a display 1901 of the personal computer. An operator holds, by one hand, an 
object tool 1902 corresponding to the display of the virtual pet and holds, by the other hand, 
various editing tools 1903. The operator operates the editing tool 1903 with respect to the object 
tool 1902 to execute various processing. 

For example, a microphone tool may be configured as the editing tool. When the 
microphone tool is caused to be close to the object tool 1902, the displayed pet outputs voice 
through a speaker. This is realized by executing a subprogram set in correspondence with the 
microphone tool such that when the distance between the microphone tool and the object tool 
1902 is below a predetermined value, processing is executed to output voice. Further, a setting 
may included that executes processing to change the tone of voice on the basis of the orientation 
information of the object tool. 

In an embodiment, various tools such as kennel, feed, and the like are set as the editing 
tool to run subprograms to carry out various processing in accordance with the various types of 
tools. 
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As, for example, processing programs shown in Figures 8-16 in the three-dimensional 
model processing apparatus of this invention, a program stored in the program memory 102 
shown in Figure 2 may be applied, or a configuration may be employed in which a program is 
stored in various memory media such as CD-ROM, CD-R, CD-RW, DVD-RAM, DVD-RW, 
DVD+RW, MO, hard disk, floppy disk, the like and combinations thereof. For example, in an 
embodiment of goods being sold by a goods provider, a processing program may be stored in 
combination on a CD with a goods catalog to offer it to a user. Thus, the user sets the CD to 
display goods on a display to execute processing using various editing tools, and a processing 
program corresponding to the selected tool is called and is executed. As a result, processing is 
carried out in accordance with the selected tool Thus, an embodiment for allowing goods 
displayed on a display to execute various operations is realized. 

As stated above, in accordance with an embodiment of a three-dimensional model 
processing apparatus and a three-dimensional model processing method of the present invention, 
there a configuration may be employed in which an operator operates various editing tools to 
execute processing determined by a corresponding tool. Accordingly, the operator can execute 
various processing with a feeling as if he actually handles or holds the displayed object. 

It should be understood that various changes and modifications to the presently preferred 
embodiments described herein will be apparent to those skilled in the art. Such changes and 
modifications can be made without departing from the spirit and scope of the present invention 
and without diminishing its intended advantages. It is therefore intended that such changes and 
modifications be covered by the appended claims. 
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ABSTRACT OF THE DISCLOSURE 
A three-dimensional model processing apparatus is provided. The three-dimensional 
model processing apparatus can more really carry out a processing change of shape, color, and 
the like with respect to a three-dimensional object being displayed on a display by allowing an 
operator to operate an object tool in which a position change and an orientation change can be 
made and an editing tool in which relative distance with respect to the object tool can be changed 
to relatively move both tools such that the relative position between the object tool and the 
editing tool is detected to execute processing corresponding to, for example, shape change or 
color processing determined by the corresponding editing tool, thereby modifying attribute data 
of the object being displayed. 
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In the Claims: 

On page 36, cancel line 1, and substitute the following left-hand justified heading 
therefor: 

CLAIMS 

Please cancel claims 1-13, without prejudice, and substitute the following claims 
therefor: 

1 4. A three-dimensional model processing apparatus comprising: 
display means for displaying an object; 

an object tool representing the displayed object, wherein the object tool is adapted to be 
manipulated in three dimensions such that the position and orientation of the object tool can be 
changed; 

an editing tool adapted to be manipulated in three dimensions such that a relative position 
between the editing tool and the object tool can be changed; and 

processing means for detecting relative position information corresponding to the relative 
position between the editing tool and the object tool, and executing processing determined by the 
editing tool to modify attribute information of the displayed object on the basis of the detected 
relative position information, thereby altering the appearance of the displayed object. 

15. The three-dimensional model processing apparatus as claimed in claim 14, 
wherein the processing means is configured to detect position information of the object tool 
based on changes in the position and the orientation of the object tool to modify the attribute 
information of the displayed object on the basis of the detected position information, thereby 
altering the appearance of the displayed object. 

16. The three-dimensional model processing apparatus as claimed in claim 14, 
wherein the processing means is configured to executing processing corresponding to a plurality 
of editing tools. 

17. The three-dimensional model processing apparatus as claimed in claim 14, 
wherein the relative position information includes at least one of a relative distance between the 
object tool and the editing tool and a relative angle between the object tool and the editing tool. 
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18. The three-dimensional model processing apparatus as claimed in claim 14, 
wherein the attribute information of the displayed object is at least one of shape, color and sound. 

19. The three-dimensional model processing apparatus as claimed in claim 14, 
wherein the processing means is configured to execute a functional operation of the displayed 
object as processing determined by the editing tool. 

20. A three-dimensional model processing method for executing various processing, 
the three-dimensional model processing method comprising the steps of: 

providing an object displayed on a display means; 
| providing an object tool representing the displayed object, wherein the object tool is 

adapted to be manipulated in three dimensions such that the position and orientation of the object 
tool can be changed; 

providing an editing tool adapted to be manipulated in three dimensions such that a 
^ relative position between the editing tool and the object tool can be changed; 

detecting relative position information corresponding to the relative position between the 
object tool and the editing tool; 

executing processing determined by the editing tool to modify attribute information of the 
displayed object on the basis of the detected relative position information; and 

updating the appearance of the displayed object based on the modified attribute 
information. 

21. The three-dimensional model processing method as claimed in claim 20, the 
method further comprising the steps of: 

detecting position information of the changed object tool based on changes in the position 
and the orientation of the object tool; 

modifying the attribute information of the displayed object based on the position 
information; and 

updating the appearance of the displayed object based on the modified attribute 
information. 
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22. The three-dimensional model processing method as claimed in claim 20, wherein 
the editing tool provided is selected from a plurality of editing tools. 

23. The three-dimensional model processing method as claimed in claim 20, wherein 
the relative position information includes at least one of a relative distance between the object 
tool and the editing tool and a relative angle between the object tool and the editing tool. 

24. The three-dimensional model processing method as claimed in claim 20, wherein 
the attribute information of the displayed object is at least one of shape, color and sound. 

25. The three-dimensional model processing method as claimed in claim 20, the 
method further comprising the step of executing a functional operation of the displayed object as 
processing determined by the editing tool. 

26. A computer readable medium storing a computer readable program for providing 
three-dimensional model processing, the computer readable program comprising the steps of: 

providing an object displayed on a display means; 

providing an object tool representing the displayed object, wherein the object tool is 
adapted to be manipulated in three dimensions such that the position and orientation of the object 
tool can be changed; 

providing an editing tool adapted to be manipulated in three dimensions such that a 
relative position between the editing tool and the object tool can be changed; 

detecting relative position information corresponding to the relative position between the 
object tool and the editing tool; 

executing processing determined by the editing tool to modify attribute information of the 
displayed object on the basis of the detected relative position information; and 

updating the appearance of the displayed object based on the modified attribute 
information. 
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REMARKS 



The present amendment makes editorial changes and corrects typographical errors in the 
specification, which includes the Abstract, in order to conform the specification to the 
requirements of United States Patent Practice. No new matter is added thereby. 

Attached hereto is a marked-up version of the changes made to the specification and 
claims by the current amendment. The attached page is captioned " Versions with Markings to 
Show Changes Made," 

In addition, the present amendment cancels original claims 1-13 in favor of new claims 
14-26. Claims 14-26 have been presented solely because the revisions by crossing out 
underlining which would have been necessary in claims 1-13 in order to present those claims in 
accordance with preferred United States Patent Practice would have been too extensive, and thus 
would have been too burdensome. The present amendment is intended for clarification purposes 
only and not for substantial reasons related to patentability pursuant to 35 U.S.C. §§101, 102, 
103 or 112. Indeed, the cancellation of claims 1-13 does not constitute an intent on the part of 
the Applicants to surrender any of the subject matter of claims 1-13. 
I Respectfully submitted, 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 
In the Specification: 

The Specification of the present application, including the Abstract, has been amended as 
follows: 

DESCRIPTION SPECIFICATION 
Throo Dimensional Model Proc e ssing Apparatus, Throo Dimensional Model Processing Method 
nnH Prnftmtn Providing Medium TITLE OF THE INVENTION 
Technical Field THREE-DIMENSIONAL MODEL PROCESSING APPARATUS, THREE- 
DIMENSIONAL MODEL PROCESSING METHOD AND PROGRAM PROVIDING 

MEDIUM 

Thin invontinn relates to a BACKGROUND OF THE INVENTION 
This present invention generally relates to three-dimensional model processing 
v apparatus, . More specifically, the present invention relates to a three-dimensional model 
I processing apparatus, a three-dimensional model processing method and a program providing 
medium for impl e menting processing such as deformation, coloring etc. to three dimensional 

I 

modd displayed display nf pnrnnnnl computer, etc.. and more particularly to a implementing 
three-dimensional model processing ^apparatus, a three dimensional model processing method 
and a program providing medium which can change various attributes such as shape or color, 
otc. of thre e dimensional model displayed on display in accordance with operation of operator to 
display it as the result of the fact that operator operates object tool as virtual object 
corresponding to throo dimensional model displayod on display and various editing tools such as 
a shapo change tool, a coloring tool, etc. 



1 



Background Art 

In the field of computer graphics, there various techniques have been developed and 
realized various techniques with respect to apparatuses or methods for displaying a three- 
dimensional graphic on a display to execute chang e processing ef to modify the orientation 
shape, eter or the like of the displayed three-dimensional graphic. As display technique onto 
display of three dimensional object, there is One such technique includes a method of inputting 
shape data of an object including depth data by using, for example , a mouse or a two- 
dimensional tablet^-ete. Further, there is also used a method of acquiring data of actual three- 
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dimensional objects by a three-dimensional digitizer or a three-dimensional scanner to acquire 
display data on the basis of that data to display such display data on display . In addition, as a 
method of carrying out processing such as deformation^-eter of an object displayed on a display, 
there is known a configuration using an input unit comprised of compose d of a glove type 
manipulator of tho glove type . 

An example of the configuration of a shape change processing apparatus of display object 
using the manipulator of using the glove type manipulator is shown in HGr Figure 1 . In ££Gr 
Figure 1, an operator fits or attaches dons a head mount mounted display 10 to observe m a 
display object 30 on display . Further, a glove type manipulator 1 1 of tho glovo type is fitted over 
or attached at hand of operator, to the operator's hand. 

Pressure s e nsor sensors or magnetic sensor sensors for detecting movement of 
movements of the operator's hand and finger is fingers are attached at to the manipulator 1 lraad 
movoment of hand of oporator is detected . A detection signal is inputted input to a control unit 
20 through an I/O interface 22. A CPU 21 executes processing in accordance with a program 
stored in a ROM 24 or a RAM 25, and executes change processing , etc. of to modify display 
paramet e r parameters relating to the display object 30 in accordance with the detection signal 
generate now thro e dimensional shape data to display, on the head mount display 10, three 
Himntinirmnl nhjnnt 10 hnnori on now display paramete r . Display object 30 is u p dated based on 
the new display parameters through display control means 23. 

For example, in the case where when an operator executes an operation to shave "shave" 
the side surface of the throo dimensional display object 30 on display by using the glove type 
manipulator 1 1 of tho glove type^ sensor of attached to the manipulator 1 1 detects the shaving 
operation of the manipulator 1 1, and the CPU 21 changes the display paramotor of tho displayed 
parameters relating to the display object 30 on the basis of dotoction signal inputted through the 
I/O 22 to display, on the head mount display 10, displayed object in which shavod portion 31 is 
form e d, the detection signal input through the I/O interface 22. Once the display object 30 is 
updated, the head mounted display 10 displays the display object 30 including the shaved portion 
3L 

As stated above, in conventional three-dimensional information input modeling systems, 
various input techniques are used. In fee a previously described processing system using mouses 
a mouse or a two-dimensional tabl e ts, since such tablet, processing is input proc e ssing by using 
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two-dimensional information , ther e exist restriction and/o r . Therefore, a feeling of restriction 
and/or disagreement may exist with respect to the operation of the three-dimensional object. 
Further, various processing such as movement, deformation, cutting, eter and the like with 
respect to the displayed object must be carried out by using only a single editing tool, erg^ 
mouGQ, r e sulting in the problem that operator is difficult to intuitively grasp for example, a 
mouse. This may result in the operator having difficulty intuitively grasping the proper setting of 
the editing tool. 

Inputs by Input data from a three-dimensional digitizer and/or a three-dimensional 
scanner are useful for input of entering the shape of virtual objects, but are not suitable for 
deformation processing , etc. of inputtod objects. Such input units arc expensive. 

Input processing by the manipulator of the glove typo is input processing which can be 
intuitively processed by and the like of three-dimensional objects. Furthermore, such input units 
are very expensive. 

It is desirable that input processing resulting from manipulation of a glove type 
manipulator is intuitive to the operator. However, in practice, any initial sotting initialization 
processing is required before actual manipulation processing , e.g., carrying out may be carried 
out. For example, a "push" operation or a "pull" operation, with respect to the object, e* 
carrying out "pull" oporation with rospoct thoroto, and thoro is thoroforo tho drawback that 
handling is may be required to initialize the settings of the glove type manipulator. This may 
result in a drawback that operating the glove type manipulator may be difficult for a user who is 
unfamiliar with the glove type units. These Moreover, these glove type input units are also 
considerably oxpensiv e . very expensive. 

Dinclnnure of tho Invention SUMMARY OF THE INVENTION 

This inv e ntion has been made in view of the problems of the above described prior art, 
and its objoct is to provide An advantage of the present invention is that it provides a three- 
dimensional model processing apparatus, a three-dimensional model processing method and a 
program providing medium in which for storing instructions for implementing a three- 
dimensional model processing method in which an operator operates respective tools in a form 
closer to more practical processing by using virtual objects and various tools corresponding to 
respective processing, thereby making it possible to execute various processing of objects such 
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as shape change, surface coloring, etc. with respect to the throo dimensional by displayed 
objects. 

This invention has been made in view of tho above mentioned task, and includes a 
display unit for throo dimonsionally displayed objects to be edited, an object tool in which 
movomont and orientation change can be mado, an editing tool in which rolativo distance with 
rospoct to tho object tool can bo changed, and processing moans for detecting rolativo position 
botwoon tho object tool and the oditing tool to oxocuto processing sot in correspondence with the 
oditing tool on tho basis of tho dotocted rolativo positional information to change attribute 
information of the objoct to bo odited displayed on the display unit. 

In this invontion, tho processing moans further includes configuration for detecting 
position information of tho changod objoct tool based on movement and orientation change of the 
objoct tool to change attribute information of tho objoct to bo edited displayed on tho display unit 
on tho basis of tho detected position information. 

Further, tho processing means includes configuration which can exocute plural different 
| processing corresponding to the kind of e diting tools. 

Further, the processing means includes configuration which executes processing set in 
correspondence with editing tool on the basis of relative position information including at least 
| one of r e lativo distance between the object tool and the oditing tool and relative angle between 
tho objoct tool and tho editing tool. 

Further, attribute information of objocts to bo edited that tho proc e ssing means changes is 
oithor one of attributo information rolating to shape, color and sound of the objoct to bo edited 
displayed on tho display unit. 

Furth e r, tho proc e ssing means includes configuration for executing functional operation 
of tho objoct to bo oditod displayed on tho display unit as processing s e t in correspondence with 
tho editing tool. 

Furthor, this invontion is dirocted to a throo dimensional modol processing method for 
oxocuting various processing with respect to the object to be odited three dimensionally 
displayed on a display unit, and this mothod includes a stop of detecting relative position 
botwoon obj e ct tool in which movement and orientation chango can be made and editing tool in 
which r e lativ e distanc e with rosp e ct to th e objoct tool can be changed, and a stop of executing 
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procoooing Got in corroopondonco with fee oditing tool on tho baoio of fee detected relative 
pooition information to chango attributo information of tho objoct to bo oditod on dioplay. 

Thio invontion Author includoG a otop of dotocting pooition information of tho changed 
objoct tool baood on movomont and orientation change of tho objoct tool, and a otop of changing 
attribute information of the object to be e ditod dioplay ed on the dioplay unit on tho baoio of the 
dotect e d pooition information. 

Thio invontion furthor includoo a otop of diocriminating a kind of tho editing tool, and 
oorvoo to oxocuto proceooing correoponding to tho kind of tho diocriminatod editing tool. 

Further, at tho otop of dotecting relative pooition, rolativo pooition including at loaot either 
ono of rolativo diotance botwoon tho objoct tool and tho oditing tool and relativo angle between 
object tool and oditing tool 

Furthor, attributo information io oithor ono of attributo information relating to ohapo, color 
and pound of object to bo edited dioplayed on dioplay meano. 

In thio invontion, functional operation of tho objoct to bo oditod dioplayod on tho dioplay 
unit io e xecuted ao procoooing sot in corr e opondenc e with the editing tool. 

In addition, thio invention io directod to a program providing medium for providing 
computer program for oxocuting, on computor oyotom, throe dimcnoional modol procoooing 
which executeo variouo proceooing with roopoct to tho object to bo edited three dimonoionally 
dioplayod on a dioplay unit, whoroin tho computor program recorded with roopoct to thio 
recording modium includeo a otop of detecting relativo pooition between objoct tool in which 
movomont and oriontation chango can bo made and tho editing tool in which rolativo diotance 
with roopect to tho objoct tool can bo changed, and a otop of executing procoooing oet in 
correopondonce with tho oditing tool on tho baoio of tho detected relative pooition information to 
chango attributo information of the objoct to bo oditod dioplayod on dioplay meano. 

Tho program providing modium adaptod go that computer program io recorded io medium 
for providing, in a form ouch that computor can road, computor program with rcopoct to, e.g., 
widoly uoed computer oyotom which can execute variouo program codoo. Ao tho medium, there 
aro rocording m e dium ouch ao CD, FD or MO, etc., and tranomiooion m e dium ouch ao network, 
etc. and thoir formo are not particularly limited. 

Such program providing medium defin e o coop e rative r e lationohip in otructuro or function 
between tho computor program and tho providing modium for realizing function of a 
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prodotorminod computor program on computer systom. In othor words, by installing computer 
program into tho computer system through the providing medium, the cooperative action is 
oxhibit e d in the computor syst e m. 

Still further objocts of this invention and more practical merits obtained by this invention 
will become moro apparont from tho description of embodiments which will bo given below. 

Brief Description of tho Drawings 

FIG. 1 is a block diagram for oxplaining conventional three dimensional model 
proc e ssing configuration using glove type manipulator. 

FIG. 2 is a block diagram showing tho configuration of a throo dimensional model 
processing apparatus according to this invention. 

FIG. 3 is a view for explaining outline of procossing in the three dimensional model 
processing apparatus according to this invention. 

FIG. 4 is a viow showing an oxamplo of the configuration in which a tablet is used in 
input unit in the three dimensional mod e l procossing apparatus according to this invention. 

FIG. 5 is a view showing an oxamplo of tho configuration in which magnotic sensors are 
used in input unit in tho throe dimensional modol procossing apparatus according to this 
invention. 

FIG . 6 is a viow showing an oxamplo of tho configuration in which ultrasonic sensors are 
usod in input unit in tho throo dimonsional modol procossing apparatus according to this 
invention. 

FIG. 7 is a viow showing an examplo of tho configuration in which ultrasonic 
transponders are used in input unit in tho thre e dimonsional model procossing apparatus 
according to this invention. 

FIG. 8 is a viow showing, as flowchart, processing in the three dimonsional model 
processing apparatus according to this inv e ntion. 

FIG. 9 is a viow showing, as flowchart, procossing in tho case whoro a trowel is used as 
the editing tool in th e thre e dimensional modol processing apparatus according to this invention. 

FIG. 10 is a vi e w showing, as data flow diagram, processing in tho case whore a trowel is 
used as tho editing tool in the throo dimensional modol processing apparatus according to this 
inv e ntion. 
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FIG. 11 is a viow showing, as flowchart, procossing in the? capo whoro a pinch io used as 
fee oditing tool in tho throo dimonGional modol processing apparatus according to this invention. 

FIG. 12 is a viow showing, ao data flow diagram, procossing in the case whoro a pinch is 
usod as tho editing tool in tho throo dimensional modol processing apparatus according to this 
inv e ntion, 

FIG. 13 is a viow showing, as flowchart, procossing in tho caso where a brush is used as 
tho oditing tool in tho throo dimensional modol processing apparatus according to this invention. 

FIG. 11 is a viow showing, as data flow diagram, procossing in tho caso whoro a brush is 
usod as tho oditing tool in tho throo dimensional modol procossing apparatus according to this 
inv e ntion. 

FIG. 15 is a viow showing, as flowchart, processing in tho case whero a spray is used as 
tho oditing tool in tho throo dimonsional modol processing apparatus according to this invention. 
I FIG. 16 is a viow showing, as data flow diagram, processing in the caso where a spray is 

usod as tho oditing tool in tho throo dimonsional model procossing apparatus according to this 
invention. 

FIG. 17A shows action area in tho case whore a spray is used as the editing tool in the 
throo dimonsional model procossing apparatus according to this invention, and FIG. 17B is a 
viow showing coloring rango in the caso whore spray is used as editing tool. 

FIGS. 18 A, 18B and ISC are views for explaining moro practical examples of the 
configuration using the throo dimonsional modol processing apparatus according to this 
inv e ntion. 

FIG. 19 is a view for e xplaining a more practical example of the configuration using the 
throo dimensional model processing apparatus according to this invention. 

Best Modo for Carrying Out the Invention 

This invention will now b e described with refer e nce to the attached drawings. 
FIG. 2 shows a block diagram showing an embodim e nt of a three dimensional model 
procossing apparatus according to this invention. The throo dimensional model processing 
apparatus 100 of this ombodimont includes central procossing unit (CPU) 101 for executing 
procossing program, program memory 102 into which procossing program is storod, data 
mcmrny 1 M *b whiefe prnnnr.ning Hntn and information such as attribute and the like. 
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To this end, in an embodiment according to the present invention, a three- dimensional 
model processing apparatus is provided. The three-dimensional model pr ocessing apparatus 
includes a display unit for three-dimensionallv displaying an ob ject and an obiect tool 
representing the displayed obiect. wherein the object tool is capable of three-dimensional 
movement such that the position and the orientation of the obiect to ol can be changed. Also 
included are an editing tool in which relative distance with respect to the obiect tool can be 
changed and a processor for detecting the relative position between the object tool and the 
editing tool. Preferably, the processor is configured to execute processi ng determined by the 
editing tool to modify attribute information of the displayed object on the basis of the detected 
relative positional information. 

Tn an embodiment, the processor is further configured to detect position in formation of 
the obiect tool based on changes in the position and the orientation of the obiect tool. Preferably. 
this detected position information is used to modify the attribute inform ation of the displayed 
obiect. In an embodiment, the processor is further configured to execute processing 
corresponding a number of different types of editing tools. In an embod iment, the detected 
relative position information includes at least one of the relative distan ce between the obiect tool 
and the editing tool and the relative angle between the object tool and the editing tool. 
Preferably, the attribute information of the displayed obiect is attribute informa tion relating to at 
least one of shape, color and sound. Further, the processor is configur ed to execute functional 
operation of the displayed object as processing determined b v the editing tool. 

Tn another embodiment according to the present invention, a three-dimensional model 
processing method is provided. Also provided are an obiect that is th ree-dimensionallv 
displayed on a display unit, an obiect tool representing the displayed obiect and an editing tool. 
The obiect tool is capable of three-dimensional movement such that the position and the 
orientation of the obiect tool can be changed and the editing tool is capable of three -dimensional 
movement such that the relative position between the editing tool and the obi ect tool can be 
changed. The three-dimensional model processing method includes the steps of detecting the 
relative position between the obiect tool and the editing tool and executing processing 
determined bv the editing tool to modify attribute information of the displayed obiect on the 
basis of the detected relative position information. 
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In an embodiment, the three-dimensional model processing method furth er includes the 
steps of detecting position information of the changed object tool base d on changes in the 
position and the orientation of the object tool and modifying the attribu te information of the 
displayed object based on the detected position information. In an embod iment, the three- 
dimensional model processing method further includes the steps of sele cting the editing tool 
from a plurality of different types of editing tools and executing processing correspo nding to the 
type of editing tool selected. 

In an embodiment, detected relative position information is at least one of the relative 
distance between the object tool and the editing tool and the relative angle between object tool 
and editing tool. In an embodiment, the attribute information of the displayed object is attribute 
information relating to at least one of shape, color and sound. In a further emb odiment, the 
three-dimensional model processing method includes the step of executing function al operation 
of the displayed object as processing determined by the editing tool. 

In vet another embodiment according to the present invention, a computer readable 
medium storing a computer readable program is provided. Also provided are an object that is 
three-dimensionallv displayed on a display unit, an object tool representing the disp layed object 
and an editing tool. The object tool is capable of three-dimensional movement such that the 
position and the orientation of the object tool can be changed and the editing tool is capable of 
three-dimensional movement such that the relative position between the editing tool and the 
object tool can be changed. The computer readable program includes the step s of detecting the 
relative position between the object tool and the editing tool and executing processing 
determined by the editing tool to modify attribute information of the displayed object on the 
basis of the detected relative position information. 

Additional features and advantages of the present invention are described in, and will be 
apparent from, the following Detailed Description of the Invention and the Figures. 

BRIEF DESCRIPTION OF THE FIGURES 

Figure 1 is a schematic diagram showing a conventional three-dimensional model 
processing configuration using a glove type manipulator. 

Figure 2 is a schematic diagram showing an embodiment of a three-dimensional model 
processing apparatus according to the present invention. 
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Figure 3 is a schematic diagram illustrating processing in a three-dimension al model 
processing apparatus according to an embodiment of the pr esent invention. 

Figure 4 is a block diagram showing an embodiment in which a tablet is used a s an input 
unit in a three-dimensional model processing apparatus according to the present invention. 

Figure 5 is a block diagram showing an embodiment in which magneti c sensors are used 
in an input unit in a three-dimensional model processing apparatus according to the present 
invention. 

Figure 6 is a block diagram showing an embodiment in which ultrasonic s ensors are used 
in an input unit in a three-dimensional model processing apparatus a ccording to the present 
invention. 

Figure 7 is a block diagram showing an embodiment in which ultrasonic transponders are 
used in an input unit in a three-dimensional model processing apparatus according to the present 
invention. 

Figure 8 is a flowchart illustrating processing in a three-dimensional model processing 
apparatus according to an embodiment of the present. 

Figure 9 is a flowchart illustrating processing where a trowel is used as an editing tool in 
a three-dimensional model processing apparatus according to an embod iment of the present 
invention. 

Figure 10 is a data flow diagram illustrating processing where a trowe l is used as an 
editing tool in a three-dimensional model processing apparatus accordi ng to an embodiment of 
the present invention. 

Figure 1 1 is a flowchart illustrating processing where a pinch is used as an editing tool in 
a three-dimensional model processing apparatus according to an embodiment of t he present 
invention. 

Figure 12 is a data flow diagram illustrating processing where a pinch is used as an 
editing tool in a three-dimensional model processing apparatus according to an embodiment of 
the present invention. 

Figure 13 is a flowchart illustrating processing where a brush is used as a n editing tool in 
a three-dimensional model processing apparatus according to an embo diment of the present 
invention. 
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Figure 14 is a data flow diagram illustrating processing where a brush is used as an 
editing tool in a three-dimensional model processing appara tus according to an embodiment of 
the present invention. 

Figure 15 is a flowchart illustrating processing where a spray is use d as an editing tool in 
a three-dimensional model processing apparatus according to an emb odiment of the present 
invention. 

Figure 16 is a data flow diagram illustrating processing where a spray is used as an 
editing tool in a three-dimensional model processing apparatus accor ding to an embodiment of 
the present invention. 

Figure 17A is a perspective view showing an action area in an embodimen t where a spray 
is used as an editing tool in a three-dimensional model proces sing apparatus according to the 
present invention, and Figure 17B is a perspective view sh owing a coloring range in an 
embodiment where spray is used as an editing tool 

Figures 18A. 18B and 18C are perspective views showing embod iments of a three- 
dimensional model processing apparatus according to th e present invention. 

Figure 19 is a perspective view showing an embodiment of a three-dimensio nal model 
processing apparatus according to the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

Figure 2 is a schematic diagram showing an embodiment of a three-d imensional model 
processing apparatus according to the present invention. The three-dimensional model 
processing apparatus 100 includes a central processing un it (CPU) 101 for executing a 
processing program, a program memory 102 into which a pr ocessing program is stored, and a 
data memory 103 in which processing data and information such as attribute data of position, 
orientation, shape, color, eter and the like of the object to be edited or attribute data of the editing 
tool, eter and the like are stored r. Also included are a frame memory 104 in which picture 
information for displaying, on a picture display unit 1 10, the object to be edited, the editing tool 
and, an instruction to user, eter and the like are stored, an input unit 105 for inputting various 
instructions with respect to the object to be edited, an external storage unit 106 for storing 
observation information or a processing result, and a bus 107 for permitting data transfer 
between these the respective units. The picture display unit 110 is connected through the frame 
memory 104. Thus, picture data stored in the frame memory 104 is maybe displayed. 
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The input unit 105 which ic charactorizod moans in this invention io constituted by plural 
groups includes a number of tools for carrying out various processing such as deformation, 
surface color change, eter and the like with respect to the object to be edited having thr e e 
flm pr-pint^i shape displayed ea fee pirtnrn display unit 1 lO. fi.e.. the object being displayed on 
the display unit or the displayed object). 

Prior to moro practical doocription of rospoctivo moans of FIG. 2, tho concept of this 
invontion will bo doocribod with roforonco to FIG. 3. Th e A three-dimensional model processing 
apparatus of this according to an embodiment of the present invention includes, as shown in FiGr 
Figure 3, an object tool 301 corresponding to the displayed object to bo oditod and editing tools 

302 to 306 for implementing various processing to the object to be oditod, which aro displayed 
on display displayed object. The object tool 301 and the editing tools 302 to 306 are also 
displayed on a display 307 . By using these tools, processing such as deformation, coloring, eter 
with roopoot to objoct 308 to bo oditod displayed on display 307 is oxocutod. and the like with 
respect to displayed object 308 may be executed. 

Tti T<in -\ ir nHitina tnnlr. thnro nro shown As shown in Figure 3. th e editing tools 302 to 
| 306 include a push tool 302 having, for example, having the shape of "trowel"a trowel for 
^ executing a "push" operation with respect to the displayed object to bo e dited, 308: a pull tool 

303 having the shape of a pinch or tongs for executing a "pull" operation with respect to the 
displayed object to bo edit e d, 308: a brush tool 304 having, erg^ for example, the shape of 
"brush" a brush for executing a "fine coloring" operation with respect to the displayed object to 

j bo odited, 308: a spray tool 305 having, e^ for example, the shape of "color spray"a can of 
sprav paint for executing a "rough coloring" operation with respect to the displayed object te-b# 
odited, 308; and a cut tool 306 having, for example, the shape of "knife"a knife for 
executing a "cut off, separate" operation with respect to the objoct to bo edited, displayed object 
308. 

Ac th e res ult "f th " feet feat np a mtnr opm-nton vnriniifi tools shown in FIG. 3 The operator 
operates the editing tools 302 to 306 in a predetermined action area fte.g.. a sensor detection 
area) as indicated by action areas 201 to 204 in FIG. 3, in moro practical sonso, carrios out 
change of Figure 3. thereby causing a change in distance between the object tool 301 and the 
editing tools 302 to 306 and a change ef in the angle therebetwee n. As a result , processing 
corresponding to the respective tools, ir©r that is . deformation of the displayed object to be edited 
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308 corresponding tojhe "push" operation by the push tool 302 and the "pull" operation by the 
pull tool 303, and/or change of the surface color of the displayed object to bo edited 308 
corresponding to the brush tool 304 and the spray tool 305 are implemented to objoct 308 to be 
nHitnH Hir.p1n.yod on display 307. the displayed object 308. 

Ac metaphor of clay model oubjcct to oporation ouch ao deformation or coloring, otc, 
input information from In an embodiment, a clav model is used as th e object tool 301 and is 
subjected to operations such as deformation, coloring, or the like. Info rmation is input from a 
sensor for detecting the position and the orientation of the clav model in the three-dimensional 
space io uood. Thio io clay objoct oonoor, and tool 301 for objoct of FIG. 3 . The object tool 301 
may be a three-dimensional shape as shown in F-IGr Figure 3, for example, a spherical shape or a 
shape approximate to the objoct to bo odit e d displayed on dioplay. This object 308. The object 
tool 301 hao tho configuration is configured such that the operator can hold it by one hand so 
such that it eaa may be freely moved or rotated. 

Moroovor, oditing tool metaphor for carrying out deformation or coloring with roopect to 
the object to be "ditH ^™ W nr. pnr.nmntm^ input information from The editing tool includes, 
as a parameter, information input from a sensor for detecting the position and die orientation of 
the editing tool in the three-dimensional space. The editing tool has a shape that operators can 
intuitively associate with a processing mode in accordance with the respective processing. 
Namoly, as ohown in FIG. 3, thoro are ahapo of "trowol" ao puoh tool 302. ohapo of "pinch" ao 
pull tool 303, ohapo of "bruoh" ao brush tool 301, ohapo of "opray" ao opray tool 305 and ohape of 
"Vnifn" nr. nut tnnl 306. otc. For example, as shown in Figure 3. the editing tool c an be in the 
shape of a trowel as a push tool 302. the shape of a pinch or tones as a pull tool 303. the shape of 
a brush as a brush tool 304. the shape of a can of sprav paint as a snrav tool 305. and the shape of 
a knife as a cut tool 306. 

ifeese The object tool 301 and the editing tools 302 to 306 correspond to the input unit 
105 in FIG. 2, and moro practical oxamplo of tho configuration of this Figure 2. An embodiment 
of the input unit will be described with reference to F4GSr Figures 4 to 7. It is to be noted that, in 
FIGS. Figures 4 to 7, the editing tools 302 to 306 are doocribod in tho citato roughly divided into a 
tool for deformation processing and a tool for color processing without finely classifying thes^ 
the type of tool being used. 
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FIG. Figure 4 is an oxample embodiment where an input unit io conotitutod with includes 
a tablet 404. Position information and orientation information of an object tool 401, a 
deformation tool 402 and a coloring tool 403 are detected by input through the input tablet 404^ 
aad. A detection data aro inputted. Dotoction function by a chip ef^a coil, or the like is added to 
the object tool 401, the deformation tool 402 and the coloring tool 403 similarly to an input pen 
associated with a conventional tablek-an d. Thus, the position and the orientation information of 
die respective tools a*e maybe detected. When For example, coils from which having different 
detection values difforont from oach othor aro obtain e d are provided on the respective surfaces 1 
to 6 of fte object tool 301 of hexahedron ao ohown in FIG. 3, for examplo, mounting surfaces on 
thn tablet -"-n Hirrrirninatod. thus shape as shown in Figure 3 . Thus, when the different detection 
values of the coils are obtained via the tablet 404. mounting surfaces are identified, thereby 
making it possible to obtain three-dimensional information from these mounting surfaces. 
Alternatively, there a configuration may be employed a configuration in which Z-axis 
information obtained from other sources, such as an optical sensor ef^a magnetic sensor orthe 
like is combined with XY axis (i.e., two-dimensional) information obtained from the tablet te 
thoroby diocriminate 404 to identify a three-dimensional position and to obtain three-dimensional 
I data. 

| FIG. Figure 5 is an e xampl e embodiment where an input unit is conotitutod by using 

includes magnetic sensors. The area within a magnetic field produced by_a magnetic 
source 504 is oauood to be an action area, and operations by an object tool 501, a deformation 
tool 502 and a coloring tool 503 are carried out within the action area. Magnetic sensors are 
respectively attached to the object tool 501, the deformation tool 502 and the coloirng tool 503, 
and tho magnotic oonsoro of tho objoct tool 501, tho doformation tool 502 and tho coloirng tool 
coloring tool 503. These magnetic sensors output magnetic displacement data to dedicated 
interface hardware 505 within the action area set within magnotic fiold producod from the 
magnotic oource 501 The dedicated interface hardware 505 calculates position information dat% 
and orientation information data of die respective tools on the basis of these this data. It is to b e 
notod that In an embodiment, the dedicated interface hardware 505 shown in FIGv Figure 5 may 
be replaced by a calculation processing program of software program. 
Figure ? 
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HGr 6 is an e xampl e embodiment where the an input unit is constituted by using includes 
ultrasonic sensors. The area where ultrasonic wave waves produced from a by an ultrasonic 
transmitting section 604 can be detected is caus e d to be an action area, and processing operations 
by an object tool 601, a deformation tool 602 and a coloring tool 603 are carried out within the 
action area. Ultrasonic sensors are respectively attached to the object tool 601 , the deformation 
tool 602 and the coloring tool 603 , and tho ultrasonic sensors of th e objoct tool 601 , the 
Hnfnrmntinn tool 602 and tho coloring tool 603 . These ultrasonic sensors detect arrival times 
from the source of an ultrasonic wave or the interference of ultrasonic waves and the like within 
the action area which is tho area whero ultrasonic wave producod from the ultrasonic transmitting 
soction 601 can bo detected dotoct arrival times from source of ultrasonic wav e or intorforence of 
ultrasonic wave, etc. Dedicated interface hardware 605 calculates position information data and 
orientation information source data of the respective tools on the basis of these the detected data. 
It is to b e noted that In an embodiment, the dedicated interface hardware 605 shown in FJGr 
^ Figure 6 may be replaced by a calculation processing program of software, software program. 
| FIG. 7 is a second example whero the input unit is constituted by using ultrasonic sensor. 

A Figure 7 is an embodiment where an input unit includes an ultrasonic sensor. An ultrasonic 
I transmitting/receiving section 704 carries out transmitting and receiving of transmits and 
receives ultrasonic wave waves . The area where ultrasonic wave waves produced #em by the 
ultrasonic transmitting/receiving section 704 can be detected is caused to be an action area^r-and 
I processin g . Processing operations by an object tool 701, a deformation tool 702 and a coloring 
tool 703 are carried out within the action area. Transponders having a function to receive 
ultrasonic wave waves produced &em by the ultrasonic transmitting/receiving section 704 and to 
send back it ultrasonic waves are respectively attached at the object tool 701, the deformation 
tool 702 and the coloring tool 703. The ultrasonic transmitting/receiving section 704 receives 
ultrasonic waves sent back from die respective tools to detect time roquirod for going and 



wave, etc. waves, or the like to calculate position information data and orientation information 
source data of die respective tools on the basis of these this data. 

The input unit 105 shown in HGr Figure 2 is realized as an input device using various 
sensors as described above. It is to be not e d that In an embodiment, information input/output 
with respect to a three-dimensional model processing apparatus 100 from sensors of the 




transmission time of the ultrasonic wav e or waves, interference of the ultrasonic 
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respective tools may be configuration through configured as a data line of wire or a wireless data 
transmitting/receiving configuration. In addition, as the configuration of an embodiment, the 
input unit 105 , th e re may be employed include a combination of the various sensors. 

As tho external otorago unit 106 shown in FIG. 2, In an embodiment, a random accessible 
memory medium such as a hard disk drive (HDD) or optical disk, oto. is dosirablo. In addition to 
tho above , an optical disk, or the like is desirable as the external storage unit 106 sh own in Figure 
2. Additionally , the external memory unit may bo constituted by include a tape streamer or non- 
volatile semiconductor memory represented bv a memory stick. In addition, Also, it is also 
possible to utilize an external storage unit of another network-connected ethef system. In an 
embodiment, the Tfee central processing unit (CPU) 101 executes processing in accordance with 
a processing program recorded in the program memory 102. 

Flew The flow of processing using the three-dimensional model processing apparatus 
I shown in FIG. 2 will now be described in accordance with the processing flow of FIG. 8. The 
processing flow show in FIG. 8 is Figure 2 will be described with reference to the flowchart 
shown in Figure 8. The flowchart shown in Figure 8 illustrates processing executed in 
I accordance with a processing program recorded in the program memory 1 02. 

In FIG. 8, tho oxamplo of tho case whoro push tool 302, pull tool 303, brush tool 301 as 
coloring tool and spray tool 305 which havo been explain e d with reference to FIG. 3 arc used as 
oditing tool will bo explained. In addition to tho above Figure 8, an embodiment will be 
explained using at least one of a push tool 302. a null tool 303. a brush tool 304 (i.e., a coloring 
tooD and a sprav tool 305 as an editing tool In an embodiment , it is possible to use various 
types of editing tools not listed above for executing various types of processing. 

Initially, at step S801 of FIG. 8 , data of a position and an orientatio n, otc. on in the three- 
dimensional space of the object to be edited are acquired. Such data aro data stored in tho data 
tnnmnry 101 r.hown in FTG, 2 and corrosponding de.. the displayed object) are acquired. This 
data corresponds to attribute data such as position, orientation, shape, color, otc. of tho object to 
nHitnH HirplnynH rm thn pi'rtnro riir.pl ay unit 1 1 0. Initial and the like of the displayed object and 
is stored in the data memory 103 f shown in Fieure 2). An initial position of the object tool 301 
{shown in FIG. 3 Figure 3) is set as a position and an orientation corresponding to these this 
attribute data. Thereafter, m the case where operator moves the object tool 301 within the action 
area which is fi.e., the area where position detection by a sensor can be made), the position and 
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the orientation of the object tool are detected by a sensor output of the object tool 301, and the 
attribute data of the displayed object is rewritten in accordance with die movement. Thus, the 
objoct to bo oditod is dioplayed appearance displayed object is updated on the picture display unit 
1 10 on the basis of the rewritten attribute data. 

Then, at step S802, whothor or not tool, i.e., a check is performed to d etermine whether 
the object tool and the editing tool ouch ao doformation, and coloring, otc. arc included are_within 
the action area is checked. The action area is eaeh die detectable area of die various sensors used 
as the input unit. Tn thn ease whoro tablet is As discussed above, various inpu t units can be used 
in accordance with various embodiments of the present invention. For example, a tablet can be 
used as the input unit? and the action area is tablet die area-r^» of the caoo whoro tablet, a 
magnetic sensor is can be used as the input unit? and the action area is the detectable area of the 
magnetic senso r. In tho caoo wh e re , or an ultrasonic sensor is can be used as the input unit? and 
the action area is die detectable area of the ultrasonic sensor. 

When it is judged determined at step S803 that the tool is not included within the action 
area, processing jumps to step S814. In tho caoo where When it is judged determined that tool is 
includod within the action area, processing proceeds to step S804 to judge determine the type or 
kind (ID) of tool being used . The judgm e nt determination of the tool kind here is judgment type 
is determined with respect to the kied type of editing teei being performed, such as deformation, 
coloring, ete and the like . With respect to the kind types of these editing tools, die operator 
himoelf may designate the tool to input identification data (ID) indicating the tool kind type, or 
may employ a configuration for outputting an identification signal (ID) from the editing tool 
oxioting in within the action area to rnrry nnt Hinorirniriation of the identify the type of tool on the 
basis of die identification signal. 

When the kind type of editing tool is judged determined at the step S804, processing 
proceeds to procoooing of steps S805 to S812. Tho proceooing from tho otep Step. S805 to the 
step S806 is illustrates processing in tho cas e where the editing tool is a push tool, tho procoooing 
from the step S807 to the step S808 is illustrate processing in th e case where the editing tool is a 
pull tool, tho procoooing from the step S809 to the step S810 is illustrate processing in tho caoe 
where the editing tool is a brush tool, and tho procoooing from th e step S811 to the step S812 is 
illustrate processing in tho caoe where the editing tool is a spray tool. The respective processing 
of the illustrated by steps S806, S808, S810, S812 a*e corresponds to subroutines corrooponding 
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te of the respective tools, and are called at tho time point when the editing tool is specified and 
are oxocutod. In tho caoo where^_Wherg the editing tool does not correspond to aay a known type 
of editing tool, processing proceeds to step S813, at which whereby a warning is produced^*** 
mnminp jr n pmnonning for . for example, displaying a warning message on the picture 
display unit 1 10 ohown in FIG. 2, or outputting a warning soundr-ete. 

Explanation will be given below with reference to FIG. 9 and figured succeeding thoreto 
Figures 9 to 16 in connection with the respective processing of the steps S806, S808, S810 T and 
S812 7 (i.e., subroutine processing corresponding to the respective editing tools). 

At step S814, die respective editing tools are displayed on the picture display unit 110, 
and display of the object to bo edited processed by the editing tool is carried out. The display 
modes (form) of die respective editing tools are modes based on shape data set ovory editing tool 
in advance for each editing tool . For example, if the editing tool is a push tool, die shape of 
"trowel" a trowel is displayed on the picture display unit 110,4f orjf the editing tool is a spray 
tool, die shape of ^a ^a can of snrav paint is displayed on the picture display unit 110. 
Moreover, the displayed object to bo oditod is displayed as an object to which deformation ef^ 
coloring or the like is applied by each editing tool. Namely, objocto to bo oditod in which 
attributed of the editing tools. Namely, as attribute data such as position, orientation, shape, 
color, otc. of tho objoot to b e oditod changod by roopoctivo editing toolo aro rofloctod are 
dioplayod. Attributoo of objocto to bo oditod rowritton by oporation by oporator of respective 
oditing toolo are and the like of the displayed object is modified bv the editing tools, these 
changes are reflected and displayed on the picture display u nit 1 10. Attribute data of the 
displayed obiect rewritten bv the operator using the editing tools is stored into the data memory 
103 (shown in FIG. 2 Figure 2) . 

Further, at step S815, oomplotion of procoooing io judgod. Complotion of proeoooing may 
hr > input frnm nnnr whn in operator, or a check is performed to determine if processing is 
completed. Processing mav be completed bv input from a user who is t he operator, or from a 
rule prescribed within the processing application. For example, in the case of with a game 
program, complotion of processing may be completed bv a completion command by such as 
game over. In addition, judgment of complotion baood on rootriction of the determination that 
processing is completed mav be based on a hardware or software restriction such as memory full T 
ete may be mrrinH nut Tn thn nnr . n whoro it. in judgod at th e and the like. When it is determined 
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at_step S815 that processing is not completed, processing returns to the start of the processing 
flow to repeatedly execute similar processing. 

Tho detail of the oubroutino procoooing ohown in FIG. 8 will now bo deooribod. FIG. 9 io 
a flow for oxplaining procoooing in tho caoo whoro puoh tool io applied to fee objoct to bo odited 
te n rr ly Hnfnrmntinn Stop SQQ1 in F igure 9 is a flowchart illustrati n g processing using a push 
tool to perform deformation of the displayed object. Step S901 is a step for obtaining 
position/orientation data on the three-dimensional space of ^gewe Pa trowel being used as a push 
tool. As previously described, such data are data obtained on the basis of respective sensor 
outputs in accordance with the kind type of input unit utilized, irSr for example , tablets, magnetic 
sensors ultrasonic sensors or the like. 

Then, step S902 io procoooing for dotormining^ _a space relationship/relative position 
I between the displayed object to bo edited and the editing tool £=e*» is determined based on the 
position/orientation data of the objoct to bo oditod which has boon acquired at tho previously 
innntinrnH rtnp S<?01 nf FTG. 8 and displayed obiect (acquired in step S80 1 of Figure 8) and the 
| position/orientation data of the trowel as the editing tool "trowel" determined at tho step 
SOftj ^obtained in step S901). 

Thon, at In step S903, judgm e nt a determination is made as to whether or not deformation 
procoGsing of tho object to bo edited uoing editing tool "trowel" io exoouted io carried out. In 
moro practical oonoo, thio judgmont oorvco to carry out of the displ ayed object by the trowel as 
the editing has been carried out. Put more simply, step 903 serv es to determine whether or not 
position data of the editing teels tool as detected by various sensors arc admittod into tho object 
tn ho oditod overlaps with position data of the object tool (i.e.. has the editing tool penetrated the 
displayed obiecf) . It is to be noted that it is not necessary that editing tools 302 to 306 a*© 
actually in tho objoct tool 301 in FIG. 3 in thio caoo, but thoro may bo omployed configuration to 
oxoouto proceooing which dotermineo that in tho caoo whoro both toolo ar e clooo oo that the 
rhntnnflo thorohotween actually penetrate the object tool 301. The appa ratus mav be configured 
to suppose that the editing has penetrated the displayed obiect if the distance b etween the obiect 
tool and the editing tool is less than a predetermined value . Alternatively, the , it io judgod that 
the e diting tool io admitted into th e objoct to be e dited. 
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Alternatively, object tool 301 may be oonotitutod with, e.g., composed of . for example, a 
deformable sponge shaped material to aloo omploy configuration such that the editing tool em 
may actually este* penetrate the object tool 301. In thin caso, more may be employed a 
configuration to oxocuto doformation of tho object to bo oditod dioplayod on tho picturo dioplay 
unit 1 1 0 nn tho basis of an embodiment, deformation of the displayed objec t mav be executed on 
the basis of the actual position data of the editing tool and the object tool in a manner caused to 
bo in correspond e nce with th e actual position data . 

Deformation processing of the object to be edited as displayed on the picture display unit 
110 at step S904 can be realized by moving positions of vertices of polygons when the display 
data is polygons data. In this case, in tho caoo where Where the distance between vertices is 
groatly apart large , processing for generating new vertices is executed. Moreover, in the case 
where the displayed object to bo odited is represented as parametric curves, their control points 
are moved. Further, in the case where the displayed object to bo e dited is represented by voxels, 
deformation can be realized by (Boolean operation) processing to add data. The deformation 
processing is carried out in accordance with data mode (form). In either case, processing based 
on displacement data of the relative distance between object tool and editing tool is carried out. 

After deformation processing of the displayed object to b e edit e d is executed at the in 
step S904, the rewritten attribute data of the displayed object to bo edited of including the 
deformed result is stored into the data memory 103 or external storage unit 106 (shown in FIG. 2 
at Figure 2) in step S905. 

In order to supplement the explanation of the flowchart shown in F-fGr Figure 9, 
explanation will be furthor given in connoction with procoooing in th e caso whoro push tool 
"trowol" is used given with reference to the data flow diagram of FIG. 10 Figure 10 using the 
trowel as a push tool . The data flow diagram is a view showing the flow of data extending from 
a data source within the object to a target within ether another object via a process for converting 
data. The data flow diagram is constituted by includes processes (indicated as ellipses) for 
converting data, data flows (indicated as arrows) for carrying data, actor objects (indicated as 
rectangles) indicating production and consumption of data, and data store objects (indication put 
between two lines) for passively carrying out storage of data. 



46 



Appl. No. PCT/JP01/03328 



The data flow of HtGr Figure 10 is illustrates processing by puoh tool "trowol", and 
procoooing using the trowel as a push tool. Processing is started by operation of the object tool 
(sensor) 1001 as an actor object and the trowel tool (sensor) 1002 as an actor object. 

From the object tool 1001, orientation information and position information of the object 
are acquired. From the trowel tool 1002, position information of the tool is acquired. 

Further, from die shape information of the object which is attribute data of the object to 
be edited (i.e.. the displayed object) which has been acquired in advance? and the surface 
attribute information of the object, the orientation information and die position information of die 
object and die position information of the tool which are stored in the data memory 103 of FIG. 
a f shown in Figure 2) . calculation processing 1003 of to calculate the degree of deformation is 
executed. On the basis of this calculation, processing for changing die attribute information? 
(i.e., deformation processing 1004) is executed. In combination with these this processing, 
rendering processing 105 ao procoooing for generating a display picture with respect to the 
picture display unit 1 10 is executed. Thus, a rendering processing result is stored in the memory 
for die display? (i.e., the frame memory 104 of FIG. 2, Figure 2) and is displayed on the picture 
display unit 110. 

FIG. 11 io a flow for explaining procoooing in tho caoo whoro pull tool "pinch" io applied 
te tb ™ ebjeet te be ™titnH tn apply Hnfhrrnntirm. Stop SI 1 01 is F igure 1 1 is a flow c hart illustrating 
processing using a pinch or tongs as a null tool to perform deformation of the disp layed object- 
Step SI 101 is a step for obtaining position data and orientation data en in the three-dimensional 
space of ^pmeh ^the pinch or tongs used as die pull tool. As previously described, such data are 
data obtained on the basis of respective sensor outputs in accordance with feied type of input unit 
utilized , i.o., tabl e ts, magn e tic censors and ultrasonic sensors. . 

Thon, at otep SI 102, whothor or not the In step SI 102. a determination is made as to 
whether or not a switch of the pinch is pushed down io judg e d. Tho pull tool "pinch" of this 
e mbodiment has the configuration in which the owitoh io provided at the pull tool "pinch"^ _The 
pinch used a pull tool in this embodiment is configured with a switch for determining the start 
point and the end point of the pull operation to s e t th e . The position of the pool tool at the fete 
point when the switch is pushed down as is either the start point ("ON") or the end point ("OFF") 
of the pull operation with respect to the displayed object to bo edited . When it is judgod at the 
determined in step SI 102 that the switch is not pushed down, processing for turning OFF "flag to 
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show it was pushed down last" is executed at step SI 109 thereafter to execute processing to 
return to the end position^-b^ (i.e., the flow start position). 

When it is judged determined at the step SI 102 that the switch has been pushed down, 
processing proceeds to step SI 103, at which point it is determined whether or not "flag to show it 
was pushed down last" is turned ON io judged. In tho case where . Where "flag to show it was 
pushed down last" is not turned "ON", it is judged that the push down of the switch ef at this 
time is "ON", i ^de., the current position is the start position of the pull operation). At step 
SI 108, processing for storing position information of the pull tool "pinch" of switch-pushing- 
down-position into the data memory 103 to return to the flow start position is executed. 

When it is judged determined at the step SI 103 that "flag to show it was pushed down 
last" is turned ON, it is judged determined that the current switch pushing down of this time is 
"OFF" and this time period is the execution period of the pull operation. In this case, at At step 
SI 104, position information of the switch pushing position? (i.e., position information as the end 
position of the pull operation) is stored into the data memory 103. 

At In the subsequent step SI 105, a movement quantity of the pinch used as the pull tool 
"pinch" from the last switch-pushing-down-point to the switch-pushing-down-point of this time is 
calculated. This movement quantity is the effective processing period of the pull operation. At 
In the subsequent step SI 106, deformation processing of the object is executed on the basis of 
the movement quantity determined at the step SI 105. At step SI 107, new attribute information 
obtained by deformation processing? (i.e., shape data of the object is stored into the data 
memory*). Display of the object is carried out on the picture display unit 1 10 on the basis of the 
new attribute information. 

For oupplomenting the description In order to supplement the explanation of the 
flowchart shown in FIGr Figure 11, explanation will be further given in connection with the case 
wh e r e th e pull tool "pinch" is us e d given w ith reference to the data flow diagram of FIG. 12. In 
FIGr Figure 12 using the pinch as a pull tool. In Figure 12, processing is started by operation of 
the object tool (sensor) 1201 as an actor object and pinch tool (sensor) 1202 as an actor object . 

From the object tool 1201, orientation information and position information of the object 
are acquired. From the pinch tool 1202, two position information of position information of the 
tool and the last position information of the tool are acquired. Th e se This information 
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correspond corresponds to the start position and the end position of the pull operation by the 
pinch or the tongs . 

On the basis of twe the acquired position information of the tool, calculation processing 
1203 of to calculate a movement quantity of the tool is executed. On the basis of the calculated 
movement quantity and position information and orientation information of the object, 
calculation processing 1204 of to calculate a deformation quantity of the object is executed. 

Further, on the basis of the calculation processing result of this die deformation quantity, 
change processing of to modify the attribute information? (i.e., deformation processing 1205} is 
executed. In combination with these this processing, rendering processing 1206 as processing 
for generating a display picture with respect to the picture display unit 1 10 is executed. Thus, a 
rendering processing result is stored into the memory for the display? (i.e., the frame memory 
104 of FIG. 2 Figure 2) and is displayed on the picture display unit 110. 

FIGr Figure 13 is a flow for explaining flowchart illustrating processing in the case where 
tke using a brush tool as a coloring tool is applied to the object to be edit e d to apply coloring to 
the displayed object . Step SI 301 is a step for obtaining position/orientation data on the three- 
dimensional space of "brush" uood ao a tool. Such the brush used as the coloring tool As 
previously describe, such data are obtained on the basis of respective sensor outputs in 
accordance with kind the type of input unit utilize d, i.o., tablets, magnetic sonsors and ultrasonic 
Gonoors ao previously described. . 

Thon : at ntop S1302. In step S1302, a space relationship/relative position between the 
displayed object to b e e dited and the editing tool is determined from die position/orientation data 
of the object to be edited which has been acquired at the previously mentioned step S801 of FIG. 
g^ttd -displaved object (acquired in step S801 of Figure 8^ and the position/orientation data of the 
editing tool "bm^" w Wn HntnrminnH nt tho Gt.op SI 301. brush as the editing tool 

(obtained in step S1301). 

Then, at stop S1303, judgment In step SI 303. a determination is made as to whether or 
not deformation processing of the displayed object to bo oditod using bv the brush as the editing 
tool "hnmh" is oxooutod is brush has been carried out. Such judgment The determination may be 
such that, ergr for example , similarly to the previously described pull tool "pinch", pinch, a 
switch is provided at the brush to detect the push-down of switch, or there the switch. 
Alternatively, a configuration may be employed such a configuration to carry out coloring by the 
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brash when the distance between the displayed object to b e edited and the editing tool which is 
dotorminod at tho otop SI 302 f determined in step S1302) is less than a predetermined distance. 

At the subsequent step SI 304, coloring processing with respect to the object is executed. 
The In an embodiment, the operator sets in advance the color used in this instance step in 
advance . After coloring processing of the displayed object to b e edit e d is executed at a the step 
S1304, rewritten attribute data of the displayed object to be edited of including the colored result 
is stored iate in the data memory 103 or the external storage unit 106 (shown in FIG. 2 at Fi gure 
2) in stepS1305. 

For supplementing In order to supplement the explanation of the flowchart shown in HGr 
Figure 13, explanation will be further given in connection with processing in the case where 
coloring tool "brush"is furth e r us e d given with reference to the data flow diagram of FIG. U 
Figure 14 using the brush as a coloring tool . In the data flow of FIGr Figure 14, processing is 
started by operation of an object tool (sensor) 1401 as actor object and a brush tool (sensor) 1402 
as an actor object . 

From the object tool 1401, orientation information and position information of the object 
| ^ required acquired . From the brush tool 1402, position information of the tool is acquired. 

Further, from the shape information of die object which is the attribute data of the 
displayed object to be edited which has been acquired in advance? and the orientation 
I information and die position information of the object and die position information of the tool 
which are stored in the data memory 103 of HGr Figure 2, calculation processing 1403 ef to 
calculate the coloring range is executed. On the basis of this calculation result and die surface 
attribute information of the object, change processing ef for changing the attribute information- 
(i.e., coloring processing 1404) is executed. In combination with these tins processing, rendering 
processing 1405 as processing for generating a display picture with respect to the picture display 
unit 110 is executed, and a rendering processing result is stored into memory for the display? 
(i.e., the frame memory 104 of FIG. 2 Figure 2) and is displayed on the picture display unit 1 10. 

FIGr Figure 1 5 is a flow for explaining flowchart illustrating processing in the case where 
the using a spray tool as a coloring tool is applied to th e object to be edited to apply coloring to 
the displayed object . Step SI 501 is step for obtaining position/orientation data in the three- 
dimensional space of "spray" used as t he spray (i.e., the spray tool having the shape of a can of 
spray paint) used as a coloring tool. As previously described, such data are data obtained on the 
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basis of respective sensor outputs in accordance with km& the type of input unit utilizeM^ 
tablets, magn e tic sensors and ultrasonic sensors. . 

Thon, at In step SI 502, the positional relationship of the action range between the 
displayed object to bo edited and the spray tool is determined from position/orientation data of 
the object to bo oditod which has boon acquired at the previously mentioned stop S8Q1 of FIG. 8 
displayed object (acquired in step S801 of Figure 8) and position/orientation data of oditing tool 
"nprny" dntnrmmnrl at the ntop S15Q1. the sprav tool as the editing tool (obtai n ed in step S1501). 

Thon : at ntep S1503. In step S1503, a determination is made as to whether or not the 
positional relationship of the action range between the displayed object to bo edited and the spray 
which has boon determined at the stop SI 502 tool (determined in step S1502) falls within the 
action area of oditing the spray tool "spray" is judged. At as the editing tool "spray" v In an 
embodiment, the area which can be colored by that the spray tool is set in advance. In order that 
For the displayed object to be edited is colored, thoro results the condition wh e re the positional 
relationship of action range between the object to bo edited displayed and the spray folte tool 
must fall within the action area of the spray tool as the editing too F'spray". In this case ^Thus, 
the action area of the spray is set as attribute of tool in advance, tool is set in advance as attribute 
data of the editing tool. 

It jg t" b" n"t"^ feat; m thn spray tnnV thn nntinn area in net as In an embodiment the 
action area of the spray tool is set as a conical area 1701 in which predetermined points of the 
spray tool are caused to be vertices as indicated by FIGr Figure 17 A, for example, and a coloring 
range 1702 with respect to an object is set within the action area on the object 1704 surface as 
shown in HGr Figure 17B. Push button 1703 is provided at on the spray tool, and the coloring 
range 1702 is set on the basis of the positional relationship at the time point when the push 
button 1703 is pushed down. 

When it is judged determined at the step SI 503 that the corresponding area is outside the 
action area of the oditing spray too r'spray" , coloring processing by the spray tool is not 
executed. As a result, processing is completed. Thus, the routine is returned to the start position. 
When it is judgod at the determined in step SI 503 that the corresponding area is inside the action 
area of the editing spray too F'spray" , the action area is displayed on the picture display unit 110 
at step SI 504. 
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Then, at In step S1505, the switch of the oditing spray tool "spray" is pushed down, and a 
determination is made as to whether or not the switch is turned ON is judged. With the fact that 
fee rvmtMi W hnnr. tnmoH ON lining nr, tho condition. . The switch being O N initiates coloring 
processing with respect to the displayed object is oxocutod at in step SI 506. ¥h@ In an 
embodiment, the operator sets in advanc e the color used in this instance step in advance . After 
coloring processing of the displayed object to bo edited is executed at in step SI 506, rewritten 
attribute data of the displayed object to bo edited of including the colored result is stored into the 
data memory 103 or the external storage unit 106 {shown in FIG. 2 at Figure 2) in step S1507. 

For supplementing tho doocription In order to supplement the explanation of the 
flowchart shown in HGr Figure 15, explanation will be further given in connection with 
procooGing in tho caoo whoro tho coloring tool "spray" is used given with reference to the data 
flow diagram of FIG. 16 Figure 16 using the spray tool as a coloring tool . In the data flow of 
52Gr Figure 16, processing is started by operation of the object tool (sensor) 1601 as actor object 
and the spray tool (sensor) 1602 as an actor object . 

From the object tool 1601, orientation information and position information of the object 
are acquired. From the spray tool 1602, position information and orientation information of the 
tool are acquired. 

On the basis of the position information and the orientation information of the spray tool, 
calculation processing 1603 ef to calculate the action area which can be colored by die spray tool 
is executed. 

Further, calculation processing 1604 ef to calculate the coloring range is carried out on 
the basis of the shape information of the object which is the attribute data of the displayed object 
to be edited which has been acquired in advance? and the orientation information and the position 
information of the object- and the calculated action area of the tool spray which are stored in the 
data memory 103 of FIG. 2, and change processing of Figure 2. Processing to modify the 
attribute information? {i.e., coloring processing 1605) is executed on the basis of this calculation. 
In combination with these tins processing, rendering processing 1606 as processing for 
generating a display picture with respect to the picture display unit 110 is executed, and a 
rendering processing result is stored into the memory for the display? (i.e., the frame memory 
104 of FIG. 2 Figure 2) and is displayed on the picture display unit 110. 
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As stated above, in accordance with the a three-dimensional model processing apparatus 
of this according to an embodiment of the present invention, the operator uses the object tool 
corresponding to the displayed object to b e edit e d and various dedicated tools to execute a 
change operation of in the relative position between both tools, whereby the processing unit 
executes processing set determined in advance in accordance with by the dedicated tool selected . 
As a result, processing corresponding to the respective editing tools such as deformation e* a 
coloring, e tc., i. e ., and the like (i.e., an attribute change) is carried out with respect to the object 
to be edited displayed on the picture dioplay unit, and object, and the display of its result is 
executed. Accordingly, the operator can execute various processing with such a feeling to 
directly implement processing to the displayed object to be edited on pictur e. 

Explanation will aew be given in connection with the more a practical example 
embodiment of an application in the of a three-dimensional model processing apparatus of this 
according to the present invention. The e xampl e where the In an embodiment, a three- 
dimensional model processing apparatus of this according to the present invention is as applied 
| to on-line shopping using a communication network such as the interneVeter or the like is shown 
in FIGS. 18Atol8C. 

I The In an embodiment, a three-dimensional model processing apparatus of this invention 

I is constituted contained within, ergr for example , personal computers of users to display, on 
I display 1801, goods from a provider who provides on-line shopping. Such goods display may be 
1 carried out by storing three-dimensional picture data relating to the goods provided from the 
j provider through a network into a storage unit such as a memory, a hard disk, etGr and the like 
within the computer of the user to read out such data^e y. Alternatively, goods display may be 
carried out by using data stored on a medium, e.g., CD or for example, CD, DVD, eter or the like 
that the goods provider provides. 

The user who studies the goods displays displayed as a three-dimensional picture ef 
goods on the display 1801 to hold holds , by one hand, the object tool (goods metaphor) 1802 
corresponding to the display goods and to hold holds, by the other hand, the operation tool (teel 
metaphor) 1803 for executing various processing with respect to the goods. User who studies A 
user studying the goods can relatively operate goods metaphor the object tool 1802 and the 
operation tool m e taphor 1803 to execute processing corresponding to respective tool metaphors 
operation tools with respect to the goods displayed on the display. For example, by changing the 
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angle/orientation of goodo motaphor the object tool 1802, the article displayed on display is 
displayed in the state where the angle/orientation is changed in a manner from HGr Figure 18A 
to FIGr Figure 1 8B. Thus, the user can observe the goods or articles from all angles. 

Further, for example, the a previously described push tool is maybe used as the operation 
tool motaphor 1803, and a push operation 4s may be executed by the tool which pushes each 
operation button portiono of goodo motaphor of the object tool corresponding to, for example, a 
video camera which io article) by tho push tool^ JThus^ as shown in HGr Figure 18C, thereby 
making it it is possible to employ such a configuration that an operation button portion of the 
video camera on the display 1801 is pushed down se such that processing corresponding to pusfc- 
down op e rations of each the depressed operation button is executed on the display. 

Thin is roalized In an embodiment, each operation button may be depressed and executed 
on the display by employing a configuration such that each switch position of the displayed 
object to be edited is specified on the basis of position information of the push tool to execute a 
processing program corresponding to die specified switch. By executing the program execution 
| corresponding to the depressed switch, operation display of the goods within displayed in display 
1801 is maybe carried out. In a manner as stated above, a user can actually obtain a feeling te of 
practically hand or hold handling or holding an article to operate and operating it by relative 
operation between goods m e taphor and tool m e taphor the object tool and the operation tool . 

Further, explanation will be given in connection with a different me*e practical example 
embodiment of an application in the of a three-dimensional model processing apparatus of this 
according to the present invention. FIGr Figure 19 constitutes the a three-dimensional model 
processing apparatus of this according to an embodiment of the present invention within, e^ for 
example, a personal computer to display a virtual pet on a display 1901 of the personal computer 
te-hei d. An operator holds , by one hand, an object tool 1902 corresponding to the display of the 
virtual pet and to hold holds , by the other hand, various editing tools 1903 to operate . The 
operator operates the editing tool 1903 with respect to the object tool 1902 corresponding to pet 
to th e r e by to_execute various processing. 

For example, th e r e may bo employed a configuration such that a a microphone tool may 
be configured as the editing tool. When the microphone tool is caused to be close to object tool 
1PQ2 corresponding p«t with thn editing tno1 1003 b e ing as microphone tool so that the object 
tool 1902, the displayed pet outputs voice through a speaker. This is realized by executing a 



54 



Appl. No. PCT/JPO 1/03328 



subprogram set in correspondence with the microphone tool in fee case where such that when the 
distance between the microphone tool and the object tool 1902 arc caused to bo cIogo so that 
distanco thorobotween is below a predetermined value to execute, , processing is executed to 
output voice. Further, a setting may bo made so as to e xecut e included that executes processing 
to change the tone of voice on the basis of the orientation information of the object tool at this 

TTTTTwT. 

In addition In an embodiment , various tools such as kennel, feed, eter and the like are set 
as the editing tool to set run subprograms to carry out oxocution in correspondence with 
respective tools so that various processing can be ex e cuted in accordance with kmd the various 
types of tools. 

As, ergr for example , processing programs shown in FIGS. S, 9, 11, 13, 15, etc. Figures 8- 
16 in the three-dimensional model processing apparatus of this invention, a program stored in the 
program memory 102 shown in ¥iG? Figure 2 may be applied, or feeee a configuration may be 
1 employed configuration in which a program is stored in various memory media such as CD- 
I ROM, CD-R, CD-RW, DVD-RAM, DVD-RW, DVD+RW, MO, hard disk or floppy disk, etc. 
For oxample, in tho case of the pr e viously goods soiling by goods provider, etc., there may be 
nmplnyr^ n rrmfi mintirm mrh that pro conning program is . floppy d isk, the like and 
combinations thereof. For example, in an embodiment of goods being sold by a goods provider, 
a processing program may be stored in combination on CD where throe dimensional data of 
| goods as goods catalog is stored to offer it to user. In this case, in tho case where the user sets a 
CD with a goods catalog to offer it to a user. Thus, the user sets the CD to display goods on a 
display to execute processing using various editing tools, and a processing program 
corresponding to the selected tool is called and is executed. As a result, processing is carried out 
in accordance with the selected tool Thus, configuration an embodiment for allowing goods 
displayed on a display to execute various operations is realized. 

As stated above, in accordance with tho throo dimonsional model processing apparatus 
and tho thr e e dimonsional mod e l processing method of this invention, there is employed the 
configuration in which operator operates various oditing tools to oxocuto processing sot in 
accordance with corresponding tool. Accordingly, the operator can execute various processing 
with a fooling as if ho hands or holds tho objoct to bo edited displayod on display to directly 
operat e it. 
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While explanation has been giv e n in connection with this invention with reference to 
specifi e d embodiments, it is obvious that th e person who is skill e d in the art can attain 
modification and/or substitution of the embodim e nts within th e scope which does not depart 
from th e gist. Namely, this inv e ntion has be e n disclosed in th e form of ex e mplification, and 
should not be r e strictively int e rpret e d. In addition, embodim e nts constituted by combining the 
above describ e d e mbodim e nts as occasion demands are also included within tho scope of this 
invention. For judging the gist of this invention, the column of the appended patent claims 
should b e taken into consideration. 

Industrial Applicability 

As describ e d above, in accordance with the three dimensional mod e l processing 
apparatus and the throo dimensional model proc e ssing method according to this invention, there 
is employed the configuration that an operator operates various editing tools to execute 
proc e ssing set in accordanc e with the corr e sponding tool. Accordingly, the operator can execute 
various proc e ssing with a f e eling as if he or she directly operat e s the object to be edited 
display e d on th e display. 

an embodiment of a three-dimensional model processing apparatus and a three- 
dimensional model processing method of the present invention, there a configuration may be 
employed in which an operator operates various editing tools to execute processing determined 
by a corresponding tool. Accordingly, the operator can execute various processing with a feeling 
as if he actually handles or holds the displayed object. 

It should be understood that various changes and modifications to the presently preferred 
embodiments described herein will be apparent to those skilled in the art. Such changes and 
modifications can be made without departing from the spirit and scope of the present invention 
and without diminishing its intended advantages. It is therefore intended that such changes and 
modifications be covered by the appended claims. 
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ABSTRACT 

A throo dimensional model processing apparatus which can mor e really carry out change 
proc e ssing of shap e , color, etc. with resp e ct to th e three dimensional object displayed on display, 
and an operator operat e s the object tool in which position change and orientation change can be 
mad e and an e diting tool in which r e lative distanc e with rospect to the object tool can be changed 
to relativ e ly move both tools so that the relative position b e tween the object tool and the editing 
tool is detected to execute processing, o.g., shape change or coloring processing, etc. set in 
correspondence with the editing tool on the basis of the dotoctod rolativo position information 
betw e en both tools to chang e shape, color, etc. by attribute change of the object to be edited 
displayed on display to display it. 

ABSTRACT OF THE DISCLOSURE 

A three-dimensional model processing apparatus is provided. The three-dimensional 
model processing apparatus can more really carry out a processing change of shape, color, and 
the like with respect to a three-dimensional object being displayed on a display by allowing an 
operator to operate an object tool in which a position change and an orientation change can be 
made and an editing tool in which relative distance with respect to the object tool can be changed 
to relatively move both tools such that the relative position between the object tool and the 
editing tool is detected to execute processing corresponding to, for example, shape change or 
color processing determined by the corresponding editing tool, thereby modifying attribute data 
of the object being displayed. 
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